Finally back on track hacking on some fun crack!

•December 30, 2007 • 1 Comment

After months of bitrotting I finally picked up the oSpy codebase during Christmas and added a new feature to oSpy2. My original plan was to get oSpy.SharpDumpLib up to a more useful level, but I got bored and decided I wanted some more useful data to analyse, and thus I ended up adding support for USB-sniffing. It’s a WDM kernel-driver (filter-driver) that intercepts the ioctls and logs them to a file. Right now it’s really simple and just dumps raw URB structs as-is, but adding the needed parsing and logging is really trivial and what I’m planning on doing next.

The cool thing is that there’s not really much USB-specific in this, so adding support for sniffing other things, for instance PCI-drivers, is quite easy.

Oh and while I’m at it, the mandatory screencast. Stay tuned!

Advertisements

Rolling your own debug symbols

•March 7, 2007 • 1 Comment

Just released oSpy 1.9.6 (changelog here). This release has a few new features, some of which are not very mature, but due to the recent critical bugfixes that went in I decided to make another release sooner rather than later. (As my time is limited I don’t feel like spending time on branching and doing things by the book. :P)

Anyway, the new release comes with a slightly obscure feature that can be very useful if you know how to use it. A lot of proprietary software is written in C++ and has internal debugging infrastructures that typically log stuff using OutputDebugString, directly to a logfile, or similar. This is obviously in order to ease debugging issues that customers are seeing, and of course also during development. Sometimes these debug messages contain stuff like “SomeClassName::SomeMethodName: shit happened, LastError=%d”. Now what’s interesting about that? It has the class name and method name in it!

This is something you’ll notice fairly often if you use oSpy together with IDA to quickly jump around in the code using the return-addresses of the API calls made by the application. Obviously you don’t want to walk through function by function, find one of these strings, visually parse out the class and method name, write down the function boundaries (and take into account chunked functions where you need to map multiple boundaries), and finally produce a list of boundaries to name mappings. No, you want to use a wonderful piece of software like Gergely Erdélyi’s idapython and write a simple little python script that does it all for you. The output is a simple .osym file that you place in the same directory as your oSpy executable, and then with a trace open you go to ‘Tools’ -> ‘Apply debug symbols’, and voila, you’re done. At that point your trace should look a lot more interesting, like for example:

oSpy’s debug symbols feature, screenshot 1oSpy’s debug symbols feature, screenshot 2

Thanks a lot to Håvard for suggesting this particular feature! 🙂

Happy reverse-engineering!

The crack-filled project has finally got a logo and a real website RSN

•March 4, 2007 • 2 Comments

I recently purchased ospy.org and thanks to Håvard the hosting seems to have been sorted out. This is a crucial part in making the collaboration feature discussed in my previous blog post a reality. What is also incredibly cool is that Matjaž Črnko (jinzo on FreeNode) and Ali Janah (lethu on FreeNode) stepped up and volunteered to do the entire web-site. Matjaž would do the engine/content and Ali would design a logo and do the web-design based on the logo.

Here’s what the logo looks like:

oSpy’s logo on a white background

The web-site is still work in progress, but here’s a small sneak-peek:

A sneak-peek of oSpy’s upcoming website

Really amazing stuff! You guys rock! 🙂

oSpy collaboration

•March 4, 2007 • 1 Comment

Olve had a really great idea for oSpy’s the other day. What if you could do a trace, create a visualization, and then go “Share it!”, enter your name/nickname and a quick description, and off it would go, straight into a central repository. As soon as it was submitted you’d get a permalink back that you could then share with your fellow hackers, and where you and others could add tags, comments etc. and help in analyzing what’s going on. At the same place you could then also have visualizations generated as HTML, SVG, PNG or whatever converters were available. People could contribute conversion scripts, like Ali recently did, and these could all be improved over time. But the thing is, wouldn’t it be wonderful to have a community of developers sharing their research, whether they’re working on open source or in-house closed source stuff that needed to inter-operate with proprietary Windows-only systems? I think this is a great idea, and it was simply impossible to let this one go by, I just had to implement it. 🙂 So here’s what it looks like:

oSpy’s collaboration feature

You can see it in action through this screencast. Thanks a lot for this great suggestion, Olve!

oSpy visualizations

•March 4, 2007 • 3 Comments

Some of you might have seen oSpy’s visualization stuff. Basically it has a few visualizers that build visualizations on top of the parsed protocol data and shows you the result in a neat timeline view with each context (usually a network connection) in a column of its own, side by side. For example:

oSpy’s visualization feature

One thing that has started to really annoy me is the fact that oSpy doesn’t run on multiple platforms. Typically, as a Linux/OS X developer working on an inter-operable replacement for some Windows-only software, you need to either have a Windows installation in dual-boot or in a virtual machine, or have someone do that for you. This is something that’s usually 5-10 minutes work, and you have what you need. The really annoying part is that usually you’ll spend hours studying the trace, and having to use a Windows-installation just for this task feels really unnecessary. So, what I envisioned was that if I could export the raw data that these visualizations were built from to XML, one could write a neat little script that would parse this and output HTML, SVG, or whatever you want. I added this feature in a hurry, and Ali took the challenge of writing a converter that would output an HTML visualization right away, and shortly afterwards he had something working, which you can see here. Check out Ali’s blog post about this for more info about this. Really great piece of work there Ali — you rock! 🙂

oSpy crypto madness

•March 4, 2007 • 1 Comment

So anyway, to start with the oldest event first, it all started after a good friend of mine, Youness Alaoui, came with a bunch of suggestions that he’d very much like to see in oSpy. His suggestions were all very good, some of them were things which I had already thought of whilst others were completely new and quite interesting. The suggestion that we ended up discussing was one I had thought of in the past, but which I was very unsure of how to solve in a clean way. This issue was around the very neat interception that’s being done with the crypto APIs, meaning that you get to see encrypted network traffic, like HTTPS, in the clear. The downside to this was that not only did you see the unencrypted traffic entering one of the crypto functions, and the encrypted data coming out, but you were also bothered with the following send call(s) transmitting the encrypted data. (And of course the other way around for receiving data.) Part of this was also the initial handshake, which is usually not very useful. So, there were basically two solutions seen:

1) Post-process the trace and coalesce these calls in a heuristical way.

2) Intercept the algorithms one level up, from within the respective API where you’ve got access to both the socket and the data where it gets encrypted/after it gets decrypted and sent/received, and log it there. Avoid logging recv/send/encrypt/decrypt for these cases by checking where these are called from.

I decided to go for the second approach, as that seemed like the best solution all things considered — less prone to error once it works. So the plan went like this: I’d start with wininet.dll’s API, as this seemed to be an API used in a lot of applications, disassemble its code with IDA, use oSpy to trace an app using this API, and then use oSpy’s IDA integration in order to conveniently examine the code where recv/send/EncryptMessage/DecryptMessage were called from. The next step was then to use oSpyAgent’s internal code signature matching API to have it scan through memory to find the spots where I thought it made the most sense to tap in (inside the C++ classes’ code behind the exported C API), and once found write some JMP-instructions, some magic trampoline code (I just love __declspec(naked) functions :-)), do the logging based on the knowledge of the stack frame, perform the instructions overwritten by the JMP, and JMP back to the next instruction after. Piece of cake, right? Yep, so it would seem. My cunning plan seemed to work, but only for a while. I did signature matching to find the return addresses of the API calls I didn’t want to do any logging for any longer, and simply constructed a blacklist that the logging code in the recv/send/EncryptMessage/DecryptMessage hooks would check the return-address against, and just avoid logging for these known addresses.

But then along came a badger, and it hit me really hard. It turned out that an internal C++ class called ICSocket was calling recv() whenever there was data to be read, and when there was secure communications involved, the object would actually be an ICSecureSocket instance, which is a class that inherits ICSocket, and this would in turn handle the data just received off the network after ICSocket had received it. A natural way to design it, but really annoying for me. If I blacklisted the recv() call made from inside ICSocket’s “oh-there’s-data-to-be-read”-method I’d effectively lose any data received for unencrypted protocols (ie. the API functions not using an ICSecureSocket socket). Hmm.. this was no good, I needed a way to tell if the object in question was an ICSecureSocket or not. So what was the easiest way to solve that problem? Well, ICSocket has virtual functions, meaning it has a vtable, meaning that if I could get hold of the this-pointer I could just look at what the pointer at offset 0 points to. If it points to the ICSecureSocket’s vtable then it was a done deal, if not then, well, just log as usual. So, the way I solved this was to simply scan for the signature of the ICSecureSocket constructor in memory, and once found read out the offset of the vtable from where it sets it on the instance being constructed. Next, just write a thin C++ class called CHookContext that keeps a dictionary of return-address to function-pointer mappings. Each hook would then just call its context’ ShouldLog() method with the return-address, and this method would then do a lookup in its dictionary. If no entry was found, just return true, everything’s good. If the function-pointer is set to NULL, return false right away, meaning you’re certain you don’t want to log for this return-address. If it’s set, call the function and decide whether to log based on the boolean it returns. Quite simple, but there was a catch. This callback would need access to the CPU registers at the point where the API function was getting called, obviously because it needed to access the this-pointer, which would either be in a register or a local variable. So this lead me to having to write another kind of hooking glue that I based off HOOK_GLUE_INTERRUPTIBLE. The result was simple and easy to use. Basically I used the PUSHAD instruction to save the contents of all general-purpose registers onto the top of the stack, and just define a C struct that would map exactly to how these were placed on the stack. Then I would just have the C callbacks declared as having instances of this struct passed by value, and voila, problem solved. The code would then look like this:

First off, instantiate the glue:
HOOK_GLUE_EXTENDED(recv, (4 * 4))

Next, declare the two callbacks:

static int __cdecl
recv_called(BOOL carry_on,
            CpuContext ctx_before,
            void *bt_addr,
            void *ret_addr,
            SOCKET s,
            char *buf,
            int len,
            int flags)
{
  ...
}

static int __stdcall
recv_done(int retval,
          CpuContext ctx_after,
          CpuContext ctx_before,
          void *bt_addr,
          void *ret_addr,
          SOCKET s,
          char *buf,
          int len,
          int flags)
{
  ...
  if (g_recvHookContext.ShouldLog(ret_addr, &ctx_before))
    {
      ...
    }
  ...
}

And finally, in the wininet hooking code:

static void *g_ICSecureSocketVTable = NULL;

static bool
ICSocketReceiveContinue_ShouldLog(CpuContext *context, va_list args)
{
  ICSocket_base *self = *((ICSocket_base **) (context->ebp - 4));
  return (self->vtable != g_ICSecureSocketVTable);
}

void
hook_wininet()
{
  ...
  if (find_signature(&signatures[SIGNATURE_ICSOCKET_RECEIVE_CONTINUE_RECV],
                     &retAddr, &error))
    {
      g_recvHookContext.RegisterReturnAddress(retAddr, ICSocketReceiveContinue_ShouldLog);
    }
  ...
}

(Please ignore the mixture of coding styles by the way. I really feel bad about this. oSpy started out with its logging agent written in C, as what it originally did was really straight-forward, but eventually as the complexity grew way beyond that there was a switch to C++, and you know the rest. This will be cleaned up eventually, but for now please just bear with me.)

As you can see it’s quite simple. But my oh my did I expect this seemingly tiny little feature request to be that much work. 🙂

Long time no pop

•March 4, 2007 • Leave a Comment

Thought it was about time I pop the recent events off my stuff-I-should-blog-about stack. A lot of stuff has happened since the last time, and I’ve been meaning to blog two or three times since then but every single time I’ve had free time on hand and felt like blogging there’s been something coming out of nowhere distracting me. That’s scatterbrainedness for you. 🙂 So, without further ado I’ll just catch up by discussing each issue in a separate blog post following this one, instead of doing one really huge post.