Chat Protocol Documentation

[some_guy]

Well I've been scouring the net for awhile looking for some winmx chat protocol documentation. It is my understanding that communications are encrypted so using a packet sniffer won't be sufficient for my purposes. I have mx sniffer and as soon as I can find a verifiably clean source for a client (just reinstalled this OS) I will take a look at that. I also have mxsock, of course. I did check out the documentation that was in japanese but was disappointed because it was basically indecipherable:P (http://2sen.dip.jp/arcs/ChatProtocol.txt)

Anyone have any tips and/or sources for good information? I'll admit that this project is the most complicated I have undertaken thus far so I am a bit nervous about importing the dll and making sure that I am using those functions properly but the header file seems to be pretty well documented with some examples. It would be helpful though if you guys could post any helpful info about any snags you ran into when using it so perhaps I can avoid them. Lightweight example code would be useful too as seeing these functions in action is what I really need to understand them properly, I think. Thank you much!

[sIrMoOn]

The protocol documentation you site is pretty old, anyway.  This would probably be a better place to start:

http://web.archive.org/web/20041027202709/www.mxtools.net/?section=techie&page=cht

That page doesn't explain the encryption process very well, but is a good reference for all of the standard (and some of the less standard) chat packets.  Note that while it labels some packets as "3.31", some as "3.53", and some as "Both", in reality some programs may mix and match, and it's best to support all of them.  The ones labeled "Extended" and "RoboMX" are only used by a few programs and not generally useful, but you may still wish to implement some of them if you think it's a nice feature for your client to have.  There's another set of extensions in the 0x34XX range, but again they only work with the right server and client.

There are several examples of the encryption floating around, but I'll overview the process real quick.  The server will send a single 0x31 byte immediately after you connect to it.  You are then required to send a 16-byte encoded block containing two encryption keys and your client type id (0x57 for a chat client).  You generate this thing using the function CreateCryptKeyId.  Neither of your two keys are used for anything.  The server responds with its own 16-byte encoded block containing two encryption keys (one for data it sends to you, and one for data you send to it) along with its client type id (which should be 0x58 for a chat server).  You extract these things using the function GetCryptKey and GetCryptKeyId.

And that's the gist of the handshake.  At that point, you send and receive chat packets according to the table I linked above, but with the caveat that everything you send out has to be encrypted (using EncryptMXTCP) and everything you receive has to be decrypted (using DecryptMXTCP).  The keys extracted using GetCryptKey are only used for the first call to EncryptMXTCP and DecryptMXTCP; the two functions will return new values to use on the next calls.

[some_guy]

I really appreciate the answer. That information has been pretty helpful but there are still some things I don't quite understand as far as how to locate cache servers, how to process information from them, how to have a client host a room,etc,etc. I did take a look at the example code that came with mxsock called roomd but I still don't understand it completely. This is actually a pretty advanced project for me....is there any lightweight example code I can work with to perhaps get a better understanding?

Also I had a question about this:
[RoomName:N][Connection:2][PrimaryIP:4][PrimaryPort:2][Files:4][UserName:N]

so for example would be like:

[RoomName:N][Connection:2][PrimaryIP:4][PrimaryPort:2][Files:4][UserName:N]

[nameofroom:N][?:2][127.0.0.1:4][2041:2][?:4][Mxuser:N]

Firstly what goes in the connection space up there? What about for files? I put a question mark in both of those because I don't understand those fields. Now when sending the messages the info would be in brackets or separated by spaces? Or is it simply that you would put the length after the colon and the servers would be able to parse it accordingly?

[GhostShip]

Hi some guy, the information needs to be in hexadecimal format and [string here :N ] means it must be followed by a "00", this is how most strings are customarily ended.
As you may notice below the string info has to be in ASCII format.

[RoomName:N][Connection:2][PrimaryIP:4][PrimaryPort:2][Files:4][UserName:N]


so you can look at this in another way

[RoomName String: S][ 00: 1 ][ Line type : 2 ][Primary IP : 4][ Primary Port : 2 ][ Shared Files Count : 4 ][ Username  String : S ][ 00 :1]

  The room(<in ASCII)     00        0008= DSL       FFFFFFFF           FFFF                FFFF                Tester(<in ASCII) 00

[sIrMoOn]

In a way, trying to talk to a cache server or host a chatroom is sort of putting the cart before the horse lol.  Since you admit this is a big project for you, I'd start off by making your client connect to a server at a known IP and port (like 127.0.0.1:3 for example if you're hosting such a room in another program) and then just see if you can talk to it.

When Quicksilver says hexadecimal, he means binary.  Quicks is not a programmer so it's not important for him to understand the difference, but you need to.  This is a binary protocol, so we're working with bytes, not characters.  Go back to the top of that page I linked and take a look at this:

Messages are in the form of:
[Type - 2 Bytes] [Length of data - 2 Bytes] [Data - Any Size]

So the message type is sent as a two-byte integer, or WORD.  Then the length of the data is sent in another WORD, and then finally the data itself.  Note once again that this all has to be encrypted before being transmitted.

Looking back at packet type 0x0064, the one you quoted.  Like every packet, it fits the format above.  The data portion looks like this:

[RoomName:N][Connection:2][PrimaryIP:4][PrimaryPort:2][Files:4][UserName:N]

The N means a null-terminated string, so it can use as many bytes as required.  The connection type is sent as a two-byte integer (WORD), the primary IP address as a 4-byte integer (DWORD), etc.  The exact values of most of these aren't so important to start with.  If you can connect to a room, do the handshake properly, and then send packet 0x0064 (the join request), then your chat server will print a "so-and-so has entered" message, and you can work from there.

[GhostShip]

When Quicksilver says hexadecimal, he means binary.  Quicks is not a programmer so it's not important for him to understand the difference, but you need to.  This is a binary protocol, so we're working with bytes, not characters

.

Unfortunately that's not at all what I mean and I find it puzzling you even suggest it.

I suggest that has something to do with the fact non programmers (it seems) call base 16 hexadecimal and binary base 2, what I stated is commonly understood by everyone else here with experience of winmx.
Btw I wasn't aware you programmed anything at the processor level Sir Moon this is new information to me, I being a non programmer had the idea you used an intermediate programming language that had nothing to do with 10101010 etc binary operations, lucky your around to put me straight.

[some_guy]

Well I figured out how to use cacheservers and whatnot. I have already written it so that the client is able to connect to a webcache, download and parse data retrieved for ip addresses, register ip with caches,etc,etc. I am still working on the parsing.

//peerscan thread
//file pointer for lolfile is already declared globally (multiple functions use this but the file is the same every time. Mostly it's opened in a+ mode but we use w+ once)
//tmpfilename1 variable is also declared globally since it doesnt change
unsigned __stdcall peerscan( void *)
{
int i, num;
char buffer[4096];//was BUFSIZ
char cacheserv1[] = "url.com"; [removed because this cache appears to be hosted on a free hosting service and thus might not be around much longer]
char cacheget1[] = "/beacon/gwc.php?ping=1&client=POPA&version=9&hostfile=1";
if (!FileExists(tmpfilename1))
{
   // DOWNLOAD FILE FROM WEBSERVER
  getfile(url512,tmpfilename1);//call download function for hardcoded url
  long lSize;
  char *buffer32, *bufferlol,*parse1[10000];;
  size_t result;
  lolfile = fopen ( tmpfilename1 , "a+" );
  if (lolfile==NULL) {fputs ("File error",stderr);}
  // obtain file size:
  fseek (lolfile , 0 , SEEK_END);
  lSize = ftell (lolfile);
  rewind (lolfile);

  // allocate memory to contain the whole file:
  buffer32 = (char*) malloc (sizeof(char)*lSize);
  if (buffer32 == NULL) ExitProcess(0);
  result = fread (buffer32,1,lSize,lolfile);
  if (result != lSize) ExitProcess(0);

  /* the whole file is now loaded in the memory buffer. */

//divide buffer into array using tokens
 
          
         
         
   parse1[0] =strtok(buffer32,"\n");

   for (int j=1;j<10000;j++)
   {
      parse1[j]=strtok(NULL,"\n");
   }


   fclose (lolfile);
}
else
{
   //LETS JUST OPEN FILE SINCE IT'S ALREADY ON DISK
   //we just open it and check all of our ips
//then if we can't connect to any we check peercache for updates

}

//after that crap we try and connect to each host one at a time
//todo: add if statement here, if we can't connect to a host we will remove
//it from the buffer and save changes to our cache with code below
//we make sure we still have CRLF at the end of each line so we can use
//our already semi-implemented method to read the file and parse it properly

   for (j = 0;j<10000;j++)
   {
        if (parse1[j])
      {
      if (parse1[j] == badip)
      {
         parse1[j] = "\n";
      }
      }
   }
      for (j = 0;j<10000;j++)
   {
        if (parse1[j])
      {
      if (parse1[j] != "\n")
      {
         strcat(bufferlol,"\r\n");
         strcat(bufferlol,parse1[j]);
      }
      }
   }


   fclose (lolfile);
   lolfile = fopen ( tmpfilename1 , "w" );
   fputs (bufferlol,lolfile);
   fclose (lolfile);



thats not the function in it's entirety but
I get the whole :   
sprintf (buffer, "GET %s HTTP/1.1\r\nHost: %s\r\n\r\n\r\n\r\n",lolurl,lolserver);
MessageBox(NULL, buffer,"DEBUG", NULL);
send (lolconnect, buffer, strlen (buffer), 0);

and I am able to parse responses and connect to peers

Still alot of work to be done and I am only a novice as you can likely tell by my techniques and I have learned alot today from reading the gnutella rfc's

[sIrMoOn]

What you're connecting to is a Gnutella cache server, not a WinMX cache server.  WinMX does not use HTTP for cache servers.

If you want to talk to a WinMX cache server, try connecting to cache1.winmxworld.com:7950.  The server will spit some encrypted data out.  Use the DecryptFrontcode function to get at the IP addresses.

[some_guy]

How do the cacheservers work? To use them I need to generate a key and everything first amirite? How can my client register with the cache servers? Do those servers share ALL of the ip's that they have cached? I know if it's anything like gnutella caches it would only keep them for an hour or so...

[sIrMoOn]

The cache server implementation has changed over time; most of the current ones are also barebones primary clients, and so are actively involved in the network; this allows them to easily discern what IP's are still good.

All the current cache servers actually ignore the key you send, if any, so you can skip that part if you'd like.  You still need to be able to check the cache server's key to determine if it's the right type, but then you use DecryptFrontCode to read the contents instead of DecryptMXTCP.

Normal sequence:

* server sends 0x31
* you send your 16-byte keyblock
* server sends you its 16-byte keyblock
* use GetCryptKey to extract the key type, initial upload key, and initial download key
* validate that the key type corresponds to someone you know how to talk to, like a secondary server
* encrypt with EncryptMXTCP and decrypt with DecryptMXTCP.  each call takes the key returned from the previous call

Cache server sequence:

* server sends 0x31
* server ignores anything you send so you can skip your keyblock
* server sends you its 16-byte keyblock
* use GetCryptKeyId to extract the key type (or GetCryptKey if you want, but the other fields of the keyblock aren't needed)
* validate that the key type corresponds to a cache server
* decrypt with DecryptFrontCode (you won't be sending anything so no need for encryption)

If cache1.winmxworld.com resolves to a machine that's not up right now, there is a more complete list of peer cache names somewhere around here.  Ask GhostShip if you can't find it.

Clear enough?