Feature Request: File I/O

[BlueSkies]

Not sure how this'll go over, as I've seen debates in the past about this subject, but that was a long time ago.

What I'd like to see in a future CVS build of MQ is the ability to manipulate files.

The benifits are obvious. Macros that read initialization data and settings from plain text files, or obtain game data that can be put to use and manipulated (like the Bazaar stuff, for example). I'm writing a path recording macro (that at the moment relies on an external VB app I wrote to parse /MQLOG output), and that would be so much easier (and more self-contained) to use if I had access to file input/output, and I think a lot of people would find this very useful for things I can't even begin to think about.

I know there are potential problems. I remember hearing talk about 'MacroQuest Trojans' that could destroy files, etc. To be honest, I don't think this is a real problem, as any such macros posted in the macro depot would be quickly identified (not to mention the fact that you shouldn't run any macro you're not familiar with).
To further reduce risk of these types of macros, we could limit file i/o to one directory, perhaps nested in the Macros directory.

Commands could be as follows:

/openfile <nameoffile> <filehandle> <openmode>
-- This would open the specified file for input/output on the specified handle, in the specified open mode (String or Binary)

$readfile(<filehandle>,<numbytes>,[<startposition>])
-- read the specified number of bytes from the specified file, starting at the specified byte position, or if the start position is omitted, starting from the current cursor position.

$readfileline(<filehandle>,[<linenumber>])
-- reads the specified line from the specified file, or if the line number is omitted, it returns the next line.

/writefile <expression> <filehandle> [<startposition>]
-- writes the specified expression to the specified file at the specified position, or if the position is omitted, it writes to the current cursor position, overwriting whatever was previously there.

/writefileline <expression> <filehandle> <linenumber>
-- writes the specified expression to the specified file at the specified line number, or if the line number is omitted, it writes to the current line number, overwriting whatever was previously there.

/closefile <filehandle>
-- Closes the file previously opened by /openfile. Filehandle could be replaced with 'All,' and would close all open files.



And we could have functions like:

$nextfile
-- Returns the next available file handle

$getpos(<filehandle>)
-- Returns the current cursor position of the specified file. For Binary files, it'd return the literal byte position, and for String files, it'd return the line number.

/setpos <filehandle> <position>
-- Sets the current cursor position of the specified file. For Binary files, it'd set the literal byte position, and for String files, it'd set the line number. Position can also be BOF (for Beginning Of File) or EOF (for End Of File).

$sizeof(<filehandle>)
-- Returns the size (in bytes for Binary files, or lines for String files) of the specified file.


An example of the use could be something like the following:

Code: Select all

Sub Main
    /openfile "somestuff.dat" 1 String
    
    /for l1 1 to $sizeof(1)
        /echo $readfileline(1,$l1)
    /next l1
    
    /closefile 1
/return

This code would open a file, assign it to File Number 1, and it'd be opened as 'String' type, so it'd be read in lines. It iterates from the first line to the last, echoing every line, and finally closes the file and exits.


That's about all I can think of right now. I'd be more than willing to help work on this, but, as I'm just starting to learn C++ at the moment (I'm a veteran VB programmer), I'd need some guidance. PM me or Email me at BlueSkies.SC@comcast.net if anyone's interested in helping me with a nudge in the right direction, or reply here if you have questions.

[Mckorr]

Ack! If all you are using is text files, and reading/writing strings from/to them, why use handles at all. How about something simpler like "writeline".

/openfile $f0 "record.txt"
:loop
/readline $f0 String
/if String=MyString goto exitloop
/nextline $f0
/goto loop
:exitloop

/writeline $f0 MyString
/closefile $f0

Basic functions: open, close, read, write, nextline, newline, stuff like that. Hmph, believe part of the function is already there, isn't there an addline command that tacks a new line onto a macro?

Anyway, skip the complicated C style syntax, manipulate strings instead of pointers.

[BlueSkies]

*shrugs* It was just a suggestion on a potential command set. :) I'm certainly willing to simplify it I literally threw those commands together as I wrote the post.

How difficult do you think this would be to work into MQ? Would it be worth the effort?

[Mckorr]

Dunno, don't have the source code here. I'd have to take a look for that addline command, see what it does and how it interfaces with the macro file.

As those who helped me with the screen location parser can attest, string manipulation is not my strong suit. I had a tremendous amount of help in getting all that working, and even then I had a tutorial open in one window, a manual open on the desk, and DKAA yelling at me about not knowing what I was doing :)

If I get a chance I'll grab the CVS here at work and see what I can find out.

[BlueSkies]

String manipulation is FUN! (*ahem*BULLSHIT*ahem*)

I just finished writing a program in VB that reads the output log file from a macro I wrote that records y,x point data and converts that into a macro that walks the path you just recorded... *phew* Fun stuff.

[/sarcasm]

I dunno. As much as I hate dealing with strings, I'm actually pretty good at it. When I get done with the pathwalking scripts (I'm about 98% done -- all I gotta do now is get it to walk the path the other way, from end to start) I'll post them, plus the VB parsing source. To be honest, I think string manipulation will be the easy part of implementing this functionality.

As for File I/O, I wasn't aware that was C-style syntax. I have no experience with C's file I/O functions.

I like your idea about creating a new variable for file handles -- the $f# stuff you have in your example code.

And about only manipulating strings -- if you look in my post, I have references to reading/writing binary data. Figured it'd be useful in some way to be able to read/write binary data. *shrugs*

[dont_know_at_all]

and DKAA yelling at me about not knowing what I was doing :)

I never yelled.

I don't see the real cost/benefit value in this proposal. Things get written to the logfile if you turn in it on. Simple perl (or vb) scripts can pull the needed data out of the log file.

We already read from files (the /macro command).

Am I missing something?

[Mckorr]

I wouldn't include binary data, too much potential for abuse, and much more open to malicious scripts. Since macros are all text scripts anyway they'd only need to be able to read/write text files.

For example your recording macro would get locations, write them to your text file. Your walking macro would then read back those text values in reverse sequence to walk you back to your starting point. Since MQ parses the macro arguments as text (you'll see a lot of szArg1, szArg2, etc. in the source code) and converts them internally as needed, using straight binary data would just confuse existing functions.

In addition, reading/writing binary data could end up turning MQ into a hacking tool far worse than it already is as people write scripts to capture binary information, then alter it and feed it back to the program. Since MQ is about automating tedious tasks, not about true "hacking", it would go against the grain of the project goals.

[BlueSkies]

I've posted a poll in MQ General to see what other users think. If it gets enough positive response, I'll tackle it myself and submit it for CVS.

McKorr, you're probably right about limiting it to String data only. Though I don't think including Binary capability would create any potential for malicious scripts, and I already posted a fix for that -- limiting what directories MQ File I/O functions have access to (like Macros/Data or something).

Doesn't matter. Binary's out -- I like straight String data anyway. Easier to work with

[BlueSkies]

Oh, and, DKAA -- I've already done the log file parsing with a VB program... It works, yes, but it's NOT a clean solution.

I'm actually about to post a set of scripts (and the accompanying VB code) that does exactly this.

[kaz]

use perl scripts to parse your log files, will be much cleaner than using mq to open files and you can use regex, seems like the most powerful and simplest solution to me, or am I missing something your trying to get out of this?

[BlueSkies]

First, I don't know perl, and I don't want to learn it.

Second, with file i/o BUILT IN to macroquest, I don't have to do all this crap manually. I can write a script that does it for me, without my interaction. MUCH more powerful than using Perl or VB or whatever.

MacroQuest is all about automation, is it not? (I'm not talking about unattended macroing here.)

What if I want to make a Bazaar macro that changes prices on the fly, starts and stops trader mode when the prices change, etc.? I can't do that automatically if I have to use some external program (in whatever language). With Built-In file processing functionality, I could. That is just ONE example. With the number of people active in the MQ community, I'm sure with this kind of functionality we'd have hundreds of uses I can't begin to think of.

So, yeah, you're missing something. The input of the entire MQ community. I'm missing it too.

Besides, we don't want those Xylo people saying "Oh yeah? We've got File I/O! Hah!" (That's a joke, guys.)

[kaz]

ok the bazaar example is valid sorta, except you wouldnt use file i/o for something like that, you'd want database i/o, and tell me how a bazaar macro that automatically changes prices doesnt sound like an unattended macro? lol

what are some other examples of how a macro would benefit from file i/o? specifics.

[BlueSkies]

Kaz, use your imagination. I'm not going to do it for you. :)

The bazaar file isn't a database, it's a plain text file. BZR_<charname>_<servernum>.ini in your EQ game directory. Check it out -- looks like any standard Windows INI file.

My pathwalking macros, which are currently in the Macro Depot, would benefit by removing the necessity to have an outside process parse the log file before you can use the path.
Right now, you have to (1)record the path, (2)leave the EQW process, (3)parse the log file, (4)enter back into the EQW process, (5)run the walk script. With File Manipulation functions, steps 2, 3, and 4 disappear completely.

I think it's pretty damned obvious what the benefits are.

[kaz]

ok I thought you meant writing a bazaar bot, which if I was going to do it would operate off a database, sorry misunderstood.

[BlueSkies]

Oh, no, hehe :) I see what you thought I meant. :)

And by changing prices on the fly, I meant via command from tells, or whatnot, so you could control your bazaar bot from another computer. I have a couple computers here set up without keyboards, mice, or monitors that I used to bot with, and something like that for the Bazaar would have been awesome.

Now that you mention it... Database access would be pretty damned cool... (though probably quite a bit overboard, hehe)