MQ Dectactable?? Of course it is....

A forum for the general posts relating to MacroQuest. *DEPRECATED: This forum is no longer in public use, but remains here for your reading pleasure. Enjoy

Moderator: MacroQuest Developers

Necromanic
orc pawn
orc pawn
Posts: 10
Joined: Sun Jan 12, 2003 11:45 pm

MQ Dectactable?? Of course it is....

Post by Necromanic » Thu Jan 23, 2003 8:35 am

Over the past few months I’ve been visiting these boards, I’ve seen more than one post on whether EQ is detectable or not. Combing my work experience with GM experience I can shed some light on the matter.

I’m not going to get into a debate as to what a ‘Guide’ or ‘Server Admin’ can or can’t do. I can’t speculate on access levels, I’ve never worked for Verant or SoE. Having said that, being a GM for other on-line games, I can say the following can and ‘IS’ being done.

Short list of GM powers…

1) Float between servers, using any name/race/class/deity combination, with or without the normal GM flag (green name tag)
2) Cast any spell available, including some spells that are not available to us mere mortals.
3) Move at any speed.
4) Zone to any Loc anywhere, in any zone.
5) Summon any item.
6) Summon new items, name them, give them stats ect.
7) Summon any mob in any zone, (to any HP’s/level) provided that mob appears on that particular zone list, i.e.. a GM will be unlikely to spawn, say a Siren in GFey.
8). Take over any mob/npc in any zone, either partially or fully. That’s to say they can take over a vendor and watch for themselves what is being bought, sold, by whom and how often, this also includes controlling any mob/npc. GM’s do not have to be on-line to do this. Think of MQ’s telnet server.
9) Chose to appear on the / or /who all gm lists. See invis has no effect. GM information can be blocked from clients just as easy as hitting an FD button. Simply put, if a GM chooses not to be seen, nothing client side can be done.
10) ‘Spectate’ on any client, think of Counter Strike or Team Fortress, while being invisible to clients.
11) Inspect, without you knowing. GM’s do not have to be stood next to you to do this, nor do they need to be in the same zone or even on-line. This information is stored in a database, requiring only a Unix client, server IP, user name and password to access.
12) Obtain client information such as total play time, session time, idle time, current zone, what you’ve killed, where, how often, and how many and where your bound ect, again, GM’s or System Admin’s do not need to be in-game to do this.
13) Take over your client, yes, they can move you. Also disconnect your keyboard, effectively blocking any commands sent from your client.

Wan’s/Lan’s differ in configuration and setup from company to company. However, observing the way EQ work’s it is possible to deduce roughly how things work.

Server’s graphical:
Domain’s such as Sullon Zek, Drinal ect, are probably made up of a number of servers. For instance, it may take 15 or more server’s clustered together to make a domain. Ever wondered how the Bazaar can come down on one server, yet not affect any other zone on the same domain? Each domain will have a load balancing, possibly redundant mirror. As a side note, I’m sceptical about SoE using a redundant mirror as we often see zones or entire domains drop out. The whole idea behind using redundant hardware is for it to kick in seamlessly if a particular piece of hardware falls over. The idea of mirroring, amongst other things is to be able to backup/restore information on a live network without impacting performance.

Server’s database:
Most likely Unix/SQL. These particular servers contain detailed database information such as spawn locations, loot tables, mobs/levels, character’s, banked items, coins, spell cast info ect ect. Unix/SQL is perfect for handling vast’s amount of information at high speed. Not only will these server’s ‘inject’ mobs/items into domains they will record and store user activities. We all know EQ mainly resides ‘server side’ how else would they know if you zoned and lost all your items or not? Or what your current jewel craft skill is (think of a proxy server). A lot of this information is useless on the face of it. I highly doubt SoE have teams of people reading through all the crap these servers generate on a daily basis, they wouldn’t need too either!

Backup:
There are a number of different backup technologies SoE could use. For instance, one approach is to use a tape library. A tape library is a box, standing approx 5ft high split in to 2 sections. The first section of tape library takes roughly up ¾ of the total space within the box, it contains a number of pigeon holes (up to 200), each ‘hole’ contains a DLT or DAT tape with a bar code on it, arranged in a semi circle. The Other ¼ of this box contains a stack of DLT or DAT backup drives, normally 6 or 8. In the middle is a robot arm with a bar code reader (laser or LED) on the end. The idea behind this piece of hardware is the ability to backup/restore a number of different servers at the same time, without having someone psychically stuff or remove tapes as and when the need requires. It is possible to backup/restore any information in any domain at any time without impacting performance or requiring human assistance.

Rest assured, every click, every key press, every move, every HP lost, every pp spent is logged and archived. Before you sit and think, hey, this guy is nuts… that’s way to much information to store, think again.. You log off with 50hp’s full mana and a SoW in Qeynos, you log back on with 50hp’s full mana and a SoW in Qeynos. All key strokes are buffered until you zone, log, loot or reach your backup timer (Sandoso saved) Backup tapes cost peanuts…

Ok, let’s put what we have learnt thus far in to practice. A few patches ago, SoE introduced a ‘combine’ throttle. How do you think they came to the conclusion a throttle was necessary? SoE have the ability to average the number of combines per month, day or even by the hour on any given domain on any given server. They do this by running a query on combines and analyse historical information, looking for peeks. They can see there is a problem, without delving into everyone’s account.

Bearing that in mind, how difficult do you think it would be to get a server to flag an account that was combing super humanly? Remember, you don’t need a human to decide something funny is going on software can do that quite easily. The very idea that SoE would mass ban accounts that showed unusual activity doesn’t make business sense. It is feasible that each user is allowed a certain number of clicks/combines per min/hour/day/month, provided you don’t reach this pre determined figure, your safe.. however… if you do, your account could be flagged. SoE may ignore 1st or 2nd flags, but as soon as you hit 3 or 4 flags alarm bells start to ring. It’s just a matter of the system emailing an Admin to make him/her aware there is a problem that requires further investigation. Completely automated, there is no need for spotty nurds to spend hours going though server logs. I would assume GM’s would be made aware of problem users and spend some time invisible, observing, taking action where/when required. On the same note, I don’t think I’d like to see hordes of paying customers being banned due to a server deciding someone is macroing. When you think about people being banned for macroing, there is no continuity. Some people macro for months without a hitch, others may turn and face and get a ban. This leads me to believe that account bans are ultimately left to customer services to decide.

It’s not a case if SoE can detect MQ or not, nor would they need to have some sort of elaborate client side detective, even if EQ were detectable. All this can be done very easily server side.

Non of the above is science fiction, it’s very real. Being a server/network Engineer I see well put together configurations like this almost on a daily basis. Most of which require very little human interaction to run.

If you use MQ, or any other macro utility, think on…. It ‘IS’ detectable…..

Necro
_______________________________

I Love it when they lie still like that....

dafyd
decaying skeleton
decaying skeleton
Posts: 6
Joined: Tue Nov 05, 2002 9:50 am

Post by dafyd » Thu Jan 23, 2003 10:22 am

I think you maybe giving them too much credence, but what do i know :P
dafyd

FlashG
Contributing Member
Contributing Member
Posts: 104
Joined: Thu Jul 11, 2002 6:38 pm

Post by FlashG » Thu Jan 23, 2003 10:31 am

HI, sounds like you have put alot of thought into speculating what the VI server can and cant do. For the most part you may be right on, what is comes down to is in your opionion VI does not need to detect the MQ client because it can be detected "from its actions". Well, thats NO news!

I find problems with item #7 because models are zone specific, and unless the model is in the "global file" it would be impossible to summon a mob that does not have a graphic. Thus no sirens in Gfay, unless SoE made it a global graphic


#6 is also questionable. yes a GM can summon "item 67523". BUT he could not rename it or give it different stats than any other "item 67
523", and this would give everyone who logged in the same modified item. A "programer" would need to create the file before the GM can summon it. From a hackers point it would be nice if item information was stored client side and a hacker could give them selves a VBD ring with 100 resists.

Lastly, The idea that every keystroke from every client is stored I find hard to believe. The logistics and wasted junk stored would be mind boggling. Also in charactor disputes GMs and not all knowing and require /report to give them info. If they were all knowing people lieing would not be possible. A few months ago SoE did change the client so it "saved" the charactor after EVERY significant action. you wont see the "saved message" but what I read said the results of the action was saved.

Thanks Flash

lifewolf
a ghoul
a ghoul
Posts: 143
Joined: Fri Oct 18, 2002 6:29 pm

Post by lifewolf » Thu Jan 23, 2003 11:02 am

Necromanic wrote:Ok, let’s put what we have learnt thus far in to practice. A few patches ago, SoE introduced a ‘combine’ throttle. How do you think they came to the conclusion a throttle was necessary? SoE have the ability to average the number of combines per month, day or even by the hour on any given domain on any given server. They do this by running a query on combines and analyse historical information, looking for peeks. They can see there is a problem, without delving into everyone’s account.
Or some GM (they probably all read these EQ hack boards) saw the C6 post, logged in EQ, tried it, and just threw a report saying it combines 30? times a second... Send that down to the programming department and you have your nurf with 0 logs involved.

Necromanic
orc pawn
orc pawn
Posts: 10
Joined: Sun Jan 12, 2003 11:45 pm

Post by Necromanic » Thu Jan 23, 2003 11:26 am

Flash,

I chose Sirens and Gfey because Sirens are Velious specific, meaning you would need the Velious expansion to see them. It would be possible for SoE to add the Siren graphic to Gfey’s spawn table, but that would require both a client (for those that didn’t have Velious) and a server side patch. Basically you said the same thing as I did.

Items are not stored as ‘files’ they are merely stored as a database record. Database records can be created in-game (live, see the Stormhammer server) or inserted directly into the database. These ‘SQL’ records contain item stats, lore information, right click graphic (also stored in a database record) equip graphic’s ect ect. This is evident some time ago when a rather good robe was linked to several server’s trade channels. This robe had something like 50int 100 to all saves and flowing thought 10, possibly linked from a GM on the test server. The only time a programmer needs to be involved is when a new item needs to be linked to a specific mob on a specific loot table for a duration of time. However, as to altering stats on existing items, this is possible, but this would have server-wide ramifications, take altering the cast time on a CoS for example. I see no reason why a GM couldn’t summon a cracked staff, up the damage, down the delay, give it saves and regen (for instance) rename this item and give it the next available item ID number and save it. I can assure you there are many items that GM’s have the ability to summon that you’ll never see.

As to ‘every’ keystroke saved… well maybe that was a little too far…. SeQ users might have a better idea of what is/isn’t sent to the server. For instance, I don’t see why pressing the ‘I’ key would need to be sent server side. As for blagging certain GM’s, GM’s sometimes need to make on the spot decisions. It’s not always possible to get that tape library cranked up to make any comparisons. They have to make on the fly decisions and go with the flow.

Lifewolf,

There are many ways for SoE to detect what you are doing. Just watching you is a good one. I didn’t want to go in to great detail and name every way they could achieve this.


Necro
_______________________________

I Love it when they lie still like that....

Malachi
a hill giant
a hill giant
Posts: 227
Joined: Tue Nov 19, 2002 1:29 am
Contact:

Hum, yeah. NO.

Post by Malachi » Thu Jan 23, 2003 12:10 pm

First we need to say that positively, 100%, there is a HUGE difference between a GM and a superuser/dev. There *has* to be.

Why? Well, let's think. With an average of something like 65,000 people logged on at one time, even if only 1 petition/bug death/drowning occurred per 10,000 users per 1 hour, that's still 156 of them a day, and you KNOW there are more than that, and you KNOW that GM's do a lot more than play milknurse to bitchy PC's and fix peoples loot. So, sure your petition may sit there for 6 hours before someone looks at it, but eventually it does. What I'm trying to say is that there are a LOT of GM's, and at least a good many of them used to be traditional players. There is NO CHANCE that verant is going to hand out...oh, let's say....500 shell accounts onto its server system whereby any guide could log in and see this/change that/whatever, because it's just too damn vulnerable. One person gets ticked off for whatever reason, and, although it would be legally actionable, they could cause hundreds of man-hours of server damage/customer service headaches. Even if they didn't do it themselves, the possibility for vulnerability at this level is rediculous.

Sure, I believe that SOME can log in and do server maintenence remotely, but I bet the numbers on this are =! than the number of GM's, and by a LOT.

Also, the concept of every keystroke being logged is rediculous. I'm sure they can look at combines per seconnd/hour/day for the server, and probably scan for your name w/i that list, however it's completely nuts that they would record everything, simply because it would be useless. It's just too much data to ever do anything with. Besides, like someone said, when you /petition they only see the last 10 lines, if they had more they'd use it. They might know how many times door 1000104035 is opened each day, but I doubt even that because it's not like they have to oil the hinges and it's wasted cycles on machines that cost far too much money to be wasting cycles on.

Besides, if you're afk macroing it's usually a lot easier to catch what you're doing based on your overall activities, not on your keypresses. For example, if you /face while sitting or track as a necro, this is obvious. Still though, Verant's not going to set up some sort of neural network to figure all this out, because it's not worth it to their business and it's not worth the cycles and bandwidth.

As always, don't be rediculous and use the program at your own risk, there is no lifeguard on duty. Don' t be rediculous though. I've said it before, this is EverQuest, not Carnivore. (or the other one, I forget what it's called just now.)

~malachi

#insert stopscaringthen00bs.h
~Oh danny boy, the pipes the pipes are calling.~

Necromanic
orc pawn
orc pawn
Posts: 10
Joined: Sun Jan 12, 2003 11:45 pm

Post by Necromanic » Thu Jan 23, 2003 1:15 pm

Malachi,

First off, I did say at the top of my post I didn’t want to get in to what a ‘Guide’ and/or a ‘Server Admin’ can/can’t do. There is only one way for you, or anyone to know what they can/can’t do and that’s to have worked for them.

Secondly, everything sent server side ‘HAS’ to be logged. How many people have gone LD just after hitting combine to come back and find ‘combined’ item attached to their cursor? On the other side of the coin, a server may go down before data can be mirrored, causing users to re-appear at their last save. Collecting large amounts of data in this manner is common practice with large corporations. The whole point of mirroring is the backup process, which they obviously have in place. It’s neither impractical nor impossible and completely automated. No human intervention is required. At worse you are looking at paying a guy to change 200 DAT/DLT tapes every quarter.

As for the text, as you know it’s buffered, in other words, stored locally. Anything stored locally would need to be streamlined to conserve memory/bandwidth and that’s why they only get 10 lines. In fact SoE may not log text at all, after all, /shout /auc /say does not have any direct impact on a server as a whole, very much unlike a combine, kill or /loot which would. On the same note, it’s not unthinkable for SoE to have separate chat servers that plug in server side to reduce processor load and log everything. Why they would want to do that is beyond me, but it IS possible.

In short, SoE’s neural net = backed up data (live, by mirroring) which can be queried at any time of the day/night in any way they like.

Necro
_______________________________

I Love it when they lie still like that....

User avatar
L124RD
Site Admin
Site Admin
Posts: 1343
Joined: Fri Jun 14, 2002 12:15 am
Location: Cyberspace
Contact:

Post by L124RD » Thu Jan 23, 2003 1:31 pm

Salutations,
Secondly, everything sent server side ‘HAS’ to be logged. How many people have gone LD just after hitting combine to come back and find ‘combined’ item attached to their cursor?
Me, Let me tell you why. Combine sent to the server. You go LD, server knows this: you are like you were before but now you have something on your cursor. So it puts it on your cursor and saves you as you dissipear. simple, and it makes more sense then logging everything, just a matter of keeping a serverside version of your current state which is overridden with every action. Not LOGGING persay, but keeping tabs on the state as to not lose as much data when an LD occurs.

Necromanic
orc pawn
orc pawn
Posts: 10
Joined: Sun Jan 12, 2003 11:45 pm

Post by Necromanic » Thu Jan 23, 2003 1:35 pm

L124RD wrote: just a matter of keeping a serverside version of your current state which is overridden with every action
Isn't that what I've been saying?

SoE can roll any character back, to any point in time from birth. Hence the backup theory. In order for roll-back to take place, it would be necessary to log everything, at least everything that could impact the server in some way.

Necro
_______________________________

I Love it when they lie still like that....

User avatar
L124RD
Site Admin
Site Admin
Posts: 1343
Joined: Fri Jun 14, 2002 12:15 am
Location: Cyberspace
Contact:

Post by L124RD » Thu Jan 23, 2003 2:01 pm

Salutations,
I am not saying that the data is LOGGED. I am saying that yes, it may be backed up, but there is quite possibly a serverside version which only keeps a current version AND DOESN'T LOG (ie outputting to a readable file). This allows them to keep a version which can be updated for REAL every save (backup?) but can also be kept in case something like an LD happens so that they do not have to roll back. If they were to actually LOG every item that happened, You could easily get a couple megs off a session, you do spewing MQ and there would be more then MQ has.

S_B_R
a lesser mummy
a lesser mummy
Posts: 72
Joined: Tue Jul 30, 2002 11:12 am

Post by S_B_R » Thu Jan 23, 2003 2:05 pm

They can't Roll back to Any point in time. they can roll back to the Any save point. a save point occurs when you see "Soandso saved", when you zone, and when you camp or go LD. for example:

If I logged in, gave you an item_A, and you gave me item_B. then we immediately trade back. Then I camped out again. There would be no way for them to roll back my account to a point when I didn't have item_A and did have item_B.

That's just a example, it would be trivial for them to roll back my account and remove my item_A and replace with your item_B, but that wouldn't be in the saved/logged data.

Also there is something very weird that happens with character deletion. I think it has to do with their backup policy. basically there's a way to get a GM to dupe ever piece of gear and PP you have, it may take a week but it can be done. Characters are backup on an individual basis, and are only backed up if they exist at the time backups are done.
[b]dd if=/dev/zero of=/dev/hda[/b]

XXXploit
decaying skeleton
decaying skeleton
Posts: 2
Joined: Wed Jan 01, 2003 3:53 pm

Post by XXXploit » Thu Jan 23, 2003 2:55 pm

Okay about the character backup policy. A short while ago my account was hacked. All items were stolen and all characters were deleted. I immediatly contacted a GM who said i would have to wait for server GM to log. Well we all know that it takes FOREVER! Finally a week or so later the Server GM contacted me. She then told me everything that had happend to my items, what time it happend, to what character on what account it was traded to and when my toons were deleted. I was told the hackers account would be banned and all my items/toons would be recovered. Took about 5 minutes for everything to be returned. Everything had been returned to the last save before my account was hacked. So all in all, everything is logged and stored.

Necromanic
orc pawn
orc pawn
Posts: 10
Joined: Sun Jan 12, 2003 11:45 pm

Post by Necromanic » Thu Jan 23, 2003 4:44 pm

L12, S_B_R….

Slight splitting of hairs here… We could argue all night about minor details such as save points, what constitutes log/backup ect.

Going back to my original post, I intended to dispel a few myths about what a GM\Server Admin\Dev (whatever) can/can’t do. The server/backup issue is only one possible scenario to literally hundreds of configurations SoE may have for their setup.

Put all my posts in a few sentences and it reads as follows…

GM’s can do lots of things *some* of you guys didn’t even know were possible. SoE have the capability to do literally anything with data flowing their way. Not only is this technically possible, but it is highly likely they have systems like this in place for just such an occasion. To put it simply, if you think just because you’ve combined 400 heady kiola’s in 4 minutes (not an exact timing for you loons that want to pick holes in that statement) and got away with it and no one will ever know… your wrong… Not all saves echo a client message, take zoning or looting as two examples.

MQ as a client side entity can not be detected. However, whatever you do with MQ on a live server certainly can be detected.

BTW, a trade would trigger a save, so it would be possible to roll you back.

Necro
_______________________________

I Love it when they lie still like that....

S_B_R
a lesser mummy
a lesser mummy
Posts: 72
Joined: Tue Jul 30, 2002 11:12 am

Post by S_B_R » Thu Jan 23, 2003 5:11 pm

A trade causes a "save" as L12 discribed. not an actual save to disk, which eventually gets backed up.
[b]dd if=/dev/zero of=/dev/hda[/b]

Requiem
decaying skeleton
decaying skeleton
Posts: 1
Joined: Thu Jan 23, 2003 5:28 pm

Post by Requiem » Thu Jan 23, 2003 5:38 pm

While somewhat on the topic, I once had a friend who had his account hacked and slashed by a rampant seq user. The gms got him his char back, couldnt get his items back, knew for a fact who did it, yet for some reason couldn't do anything but tell him who did it. Anybody want to speculate on this?
"I once asked a man whether it was ignorance or apathy which ruined the world. He replied I don't know and I don't care."