Use jmap and jhat to Detect Memory Leak

I noticed that my servers are running out of memory after one whole day’s service.

At first, I removed all functions that eat up memory. But still the next morning when the traffic is down, the memory is still high. So it seems I have memory leak in my server.

After making some searches and reading some blog articles, I learned that I can try to find view all memory by using jmap and jhat.

First, I made the dump:

jmap -dump:file=heap.bin ####

It was about 800M file, as my server is running at 1.5G maximum memory on 2G-memory server. I tried to use jhat to give a memory view:

jhat -J-mx1536m heap.bin

It run but failed with OutOfMemoryError, as my service was still running and it had no enough memory. Fortunately, I had another server without service running, so I uploaded the dump file to another server, and tried to start jhat there.? It run but with lots of warning message like the following:

WARNING:? Failed to resolve object id 0xac2f1690 for field key (signature L)
WARNING:? Failed to resolve object id 0xacd1c208 for field value (signature L)
WARNING:? Failed to resolve object id 0xac08df00 for field value (signature L)
WARNING:? Failed to resolve object id 0xac2f3f28 for field key (signature L)
WARNING:? Failed to resolve object id 0xac08df00 for field value (signature L)
WARNING:? Failed to resolve object id 0xac2eea70 for field key (signature L)
WARNING:? Failed to resolve object id 0xacd1c208 for field value (signature L)
WARNING:? Failed to resolve object id 0xac08df00 for field value (signature L)

And after a long time, it still gave me an OutOfMemoryError!

So I made another dump from another server which was running on its early stage with less memory being used. So the dump file was a lot smaller. And I could run jhat on the same without uploading to another. And after a very long time of waiting I finally got message:

Snapshot resolved.
Started HTTP server on port 7000
Server is ready.

Visiting http://###.###:7000/ , I noticed that jhat was slow. But it helped! I found my memory leak.

Great tools! I think I can reduce servers but still provide more stable services now.

This entry was posted in Java, Server. Bookmark the permalink.

3 Responses to Use jmap and jhat to Detect Memory Leak

  1. David says:

    I like your javascript compressor, but the link to the jslib @
    http://bl.ognize.com/j2s-ajax-rss-reader/net.sf.j2s.lib_1.0.0.v320/j2slib/j2slib.lz77.z.js
    is not working.

    can you repost or email it to me?
    my address is flaming.red.dingo
    it’s through gmail.com

    Thank you very much.

    PS: sorry for the off topic comment – seemed like the only place you posted recently

  2. AJ Weaver says:

    Can you please elaborate on how you found the memory leak? I have gotten as far to create the site on port 7000, but there is so much information, I dont know where to start. I dont have anything that screams “blocking thread” or “memory leak”. However I do have 50+ blocking threads and I no longer can initiate/open a socket on the server application from the client. Im thinking that all the blocking threads I have are consuming my sockets, but cant figure out a way to prove this.

    • Zhou Renjian says:

      Sorry for late reply.
      To found memory leak, you need to browse those nodes which consumes most of the memory. Try to find out who keep references to these nodes. If those nodes should release the memory-heavily-consumed nodes, try to set reference to null so JVM can release them automatically and memory leak will be fixed.