Category Archives: Java

Use jmap and jhat to Detect Memory Leak

Posted on January 8, 2011 by Zhou Renjian

I noticed that my servers are running out of memory after one whole day’s service.

At first, I removed all functions that eat up memory. But still the next morning when the traffic is down, the memory is still high. So it seems I have memory leak in my server.

After making some searches and reading some blog articles, I learned that I can try to find view all memory by using jmap and jhat.

First, I made the dump:

jmap -dump:file=heap.bin ####

It was about 800M file, as my server is running at 1.5G maximum memory on 2G-memory server. I tried to use jhat to give a memory view:

jhat -J-mx1536m heap.bin

It run but failed with OutOfMemoryError, as my service was still running and it had no enough memory. Fortunately, I had another server without service running, so I uploaded the dump file to another server, and tried to start jhat there.? It run but with lots of warning message like the following:

WARNING:? Failed to resolve object id 0xac2f1690 for field key (signature L)
WARNING:? Failed to resolve object id 0xacd1c208 for field value (signature L)
WARNING:? Failed to resolve object id 0xac08df00 for field value (signature L)
WARNING:? Failed to resolve object id 0xac2f3f28 for field key (signature L)
WARNING:? Failed to resolve object id 0xac08df00 for field value (signature L)
WARNING:? Failed to resolve object id 0xac2eea70 for field key (signature L)
WARNING:? Failed to resolve object id 0xacd1c208 for field value (signature L)
WARNING:? Failed to resolve object id 0xac08df00 for field value (signature L)

And after a long time, it still gave me an OutOfMemoryError!

So I made another dump from another server which was running on its early stage with less memory being used. So the dump file was a lot smaller. And I could run jhat on the same without uploading to another. And after a very long time of waiting I finally got message:

Snapshot resolved.
Started HTTP server on port 7000
Server is ready.

Visiting http://###.###:7000/ , I noticed that jhat was slow. But it helped! I found my memory leak.

Great tools! I think I can reduce servers but still provide more stable services now.

Posted in Java, Server | 3 Comments

Implementing Gmail-alike Web Mail Client by Java2Script

Posted on December 3, 2009 by Zhou Renjian

WeMail, this to-be-implemented Gmail-alike web mail client, is based on technologies of Java, SWT, JavaMail and Java2Script.

Before going into designing and implementing details, here are some background information.

Java2Script (J2S) Pacemaker provides an Eclipse Java to JavaScript compiler plugin and an implementation of JavaScript version of Eclipse Standard Widget Toolkit (SWT) with other common utilities, such as java.lang.* and java.util.*. You can convert your SWT-base Rich Client Platform (RCP) into Rich Internet Application (RIA) by Java2Script Pacemaker.

JavaMail is a Java API used to receive and send email via SMTP,POP3 and IMAP. JavaMail is built into the Java EE platform, but also provides an optional package for use in Java SE.

SWT is an open source widget toolkit for Java designed to provide efficient, portable access to the user-interface facilities of the operating systems on which it is implemented.

Google Web Toolkit (GWT) is an open source set of tools that allows web developers to create and maintain complex JavaScript front-end applications in Java. Other than a few native libraries, everything is Java source that can be built on any supported platform with the included GWT Ant build files.

Why implementing another Gmail-alike web mail client?

When GWT is released, lots of developers ask whether GWT will be used in Google’s major products, like Gmail, Google Reader or other products. Yeas went by, but there is still no Gmail powered by GWT. So using Java-to-JavaScript technology to build another Gmail web client should be fun for technical geeks. I choose Java2Script to build this web mail client, hoping it would help improving Java2Script toolkit.

I like Gmail UI, but I can not use its UI for other mail providers, like Hotmail or Yahoo! Mail. So I believe WeMail will help users doing their email jobs like receiving, composing and replying emails, in Gmail-alike UI for all mail providers. And it will sure increase their daily email processing efficiency.

And there are lots of companies that might block web access to Gmail, Yahoo! Mail or other mail providers. In these cases, a web mail client for general POP/IMAP & SMTP will help those users accessing emails even behind firewalls.

So I finally made up my mind to build it. And it was my two weeks’ fun.

Dang, dang, dang! Here it comes WeMail – Web Mail Client for Gmail/Hotmail/Live Mail/Yahoo! Mail/AOL Mail.

Configureing mail servers and login:

WeMail main window, listing emails:

Viewing an email:

Composing an email:

How is this Gmail alike web mail client being implemented?

I use WindowsBuilder Pro (formerly known as SWTDesigner) to design all the UI elements. Here are screenshots of its UI components being designed:

Overview of WeMail in designer:

Designing top menu bar:

Designing logo and search bar:

Designing left side widget:

Designing email selector bar:

Designing email message header:

Designing email message viewer:

Designing email composer:

Designing footer panel:

Designing login window:

Designing mail server configuring window:

Designing and adjust user interfaces causes lots of time.

And what about the receiving and sending emails? I use JavaMail to do email jobs. Between SWT UI and JavaMail, Java2Script’s Simple RPC technology is used. Those mail jobs are separated into following RPCs:
1. CheckMails
2. DiscardEmails
3. FetchMailContent
4. ListFolderMails
5. MarkMailAsRead
6. SaveDraft
7. SendMail

UI events will trigger those RPCs being called in separate threads and UI will be updates after their jobs are done.After RPCs get woven into UI elements. It is ready to be run as a native Java desktop application. So it is:

Running, debugging, testing, bug-fixing, all take times. But up until now, all I do is native Java developments. No HTML, no JavaScript technologies is needed.

After the Java desktop version of WeMail is OK, it will be ready for being converted in an AJAX based web browser application. That is the Java2Script job (I am not going into Java2Script details, more information can be found on Java2Script at Sourceforge.net.): Install Java2Script Eclipse plugin and convert existed project into a Java2Script project. All those *.java files will be compiled int JavaScript automatically. Then pack all necessary files into a Servlet *.war file and deploy it into Java Servlet container server like Tomcat. And work is done. WeMail is available at http://webuzz.im/mail/ .

So it is my WeMail’s development log. Having fun.

Posted in AJAX, Java, Java2Script | Tagged Gmail, Java, Java2Script, JavaMail, Simple RPC, WeMail | 5 Comments

Connect Yahoo IMAP Server by JavaMail

Posted on November 29, 2009 by Zhou Renjian

These days, I were implementing an online web mail client for all mail services, like Gmail, Hotmail/Live Mail, Yahoo! Mail, AOL/AIM Mail. And adding Yahoo! Mail support cost me lots of time, but the solution is simple.

Yahoo! Mail provides only web access for free users and does not provide free POP3 services. You need to buy Yahoo! Mail Plus to have POP support. Not long before, Yahoo! Mail provided IMAP services for Zoho Web Mail. It means that IMAP support is free for any Yahoo! Mail users, no Yahoo! Mail Plus is needed. But Yahoo! Mail’s IMAP services is not a totally standard IMAP service. You need send some special tokens before you login. So a customized Thunderbird is need to Yahoo! IMAP support.

JavaMail provides a convenient way to connect mail servers by POP/IMAP/SMTP and others. Here is a “how-to” of making a little modification of JavaMail library to support Yahoo! Mail’s IMAP service.

1. Download sources of JavaMail
2. Modify com.sun.mail.imap.IMAPStore.login(IMAPProtocol, String, String) from a private method into a protected method, because we need to override it for Yahoo! IMAP.
3. Replace the compiled IMAPStore*.class files in JavaMail’s binary library file mail.jar.
4. Write two classes:

public class YahooIMAPStore extends IMAPStore {

	public YahooIMAPStore(Session session, URLName url) {
		super(session, url);
	}

	@Override
	protected void login(IMAPProtocol p, String u, String pw)
			throws ProtocolException {
		Response[] r = p.command("ID (\"GUID\" \"1\")", null);
		p.notifyResponseHandlers(r);

		super.login(p, u, pw);
	}
}

public class YahooIMAPSSLStore extends IMAPSSLStore {

	public YahooIMAPSSLStore(Session session, URLName url) {
		super(session, url);
	}

	@Override
	protected void login(IMAPProtocol p, String u, String pw)
			throws ProtocolException {
		Response[] r = p.command("ID (\"GUID\" \"1\")", null);
		p.notifyResponseHandlers(r);

		super.login(p, u, pw);
	}
}

5. Modify JavaMail related codes to use overrided YahooIMAPStore:

if ("yahoo.com".equals(domain)
		|| "ymail.com".equals(domain)
		|| "rocketmail.cn".equals(domain)) {
	if (protocol == null) {
		protocol = "imaps"; host = "imap.next.mail.yahoo.com"; port = 993;

		protocol = "imap";
		host = "imap.mail.yahoo.com";
		port = 143;
	}
	if (sendProtocol == null) {
		sendProtocol = "smtps";
		sendHost = "smtp.mail.yahoo.com";
		sendPort = 587;
	}
}
...
if (host.indexOf(".yahoo.com") != -1) {
	props.setProperty("mail.imap.class", "im.webuzz.mail.YahooIMAPStore");
	props.setProperty("mail.imaps.class", "im.webuzz.mail.YahooIMAPSSLStore");
}
Session session = Session.getInstance(props, null);
if (host.indexOf(".yahoo.com") != -1) {
	session.addProvider(new Provider(Provider.Type.STORE, "imap",
		"im.webuzz.mail.YahooIMAPStore", "WeBuzz", "1.0"));
	session.addProvider(new Provider(Provider.Type.STORE, "imaps",
		"im.webuzz.mail.YahooIMAPSSLStore", "WeBuzz", "1.0"));
}

store = session.getStore(protocol);
store.connect(host, user, password);
...

Done! Enjoy Yahoo! Mail’s IMAP support by JavaMail.

BTW: You can test out WeBuzz’s Web Mail for Yahoo! Mail, which is using JavaMail and IMAP with the above hacks.

Posted in Java | Tagged IMAP, JavaMail, Yahoo Mail | 10 Comments

Implementing Web AIM Client

Posted on May 6, 2008 by Zhou Renjian

Here are some notes when I was struggling to develop a web AIM client:

1. Download AIM SDK, both Windows and Linux
In Linux environment, you may already have NSS/NSPR libraries installed, if not following instructions to install it. You only need to run:

cd dist/release/
ln -s /usr/lib/libnspr4.so .
ln -s /usr/lib/libnss3.so .
ln -s /usr/lib/libnssckbi.so .
ln -s /usr/lib/libplc4.so .
ln -s /usr/lib/libplds4.so .
ln -s /usr/lib/libsmime3.so .
ln -s /usr/lib/libsoftokn3.so .
ln -s /usr/lib/libssl3.so .
ln -s /usr/lib/libfreebl3.so .
export LD_LIBRARY_PATH=/usr/local/tomcat/lib/

2. Get a developer key to use AIM SDK. The developer web site does not accept external email account, so create a new one. In fact, more new AIM screen names are needed, as testing AIM requires it.

3. Try to compile and run samples/accjsample:

cd samples/accjsample
javac -cp ../../dist/release/accjwrap.jar AccJSample.java Prefs.java
java -cp ../../dist/release/accjwrap.jar:./ AccJSample username password developerkey

But you may get error messasge like:

UNKNOWNVALUE
com.aol.acc.AccException: IAccSession_SignOn
at com.aol.acc.AccSession.SignOn(Native Method)
at com.aol.acc.AccSession.signOn(AccSession.java:113)
at AccJSample.(AccJSample.java:47)
at AccJSample.main(AccJSample.java:22)

But if you go into dist/release/ directory, where *.so are located, and run:

/usr/local/java/bin/java -cp ../../dist/release/accjwrap.jar:../../samples/accjsample/ AccJSample username password developerkey

You may get things running correctly:

[5/5/08 8:34 AM]: OnStateChange: Connecting, ACC_S_OK
[5/5/08 8:34 AM]: OnStateChange: Validating, ACC_S_OK
[5/5/08 8:34 AM]: OnStateChange: Transferring, ACC_S_OK
[5/5/08 8:34 AM]: OnStateChange: Negotiating, ACC_S_OK
[5/5/08 8:34 AM]: OnStateChange: Starting, ACC_S_OK
[5/5/08 8:34 AM]: OnStateChange: Online, ACC_S_OK
[5/5/08 8:34 AM]: OnUserChange: ACC_S_OK
…

That is because *.so may not be loaded correctly if they are not in the current working directory.

Someone says that inaccurate time may cause problems, so run:

/etc/init.d/ntpd start

to synchronize your server’s time.

4. following, it comes to the time that developing AIM desktop client, and then converting it into Java Servlet based AIM client. So, tutorial “Creating AIM-Enabled Application in Java” series are recommended.

5. And time to deploy your web AIM client to tomcat servlet container.

If your Tomcat 6 server is on Windows, you may need to copy all dist/release/*.dll into your %CATALINA_HOME%/bin/ folder, copy dist/release/accjwrap.jar into %CATALINA_HOME%/lib and restart Tomcat services to get things done. Deploy accjwrap.jar directly into webapps folder is not recommended, because you may want to restart your servlet application frequently while loading and unloading *.dll modules may cause “already loaded in another classloader UnsatisfiedLinkError” problems for Tomcat, which may result that you need to restart your Tomcat server.

If your Tomcat 6 server is on Linux, you also need to copy all dist/release/*.so into your $CATALINA_HOME/bin/ folder, copy dist/release/accjwrap.jar into $CATALINA_HOME/lib. And you need to modify your catalina.sh and add two lines:

LD_LIBRARY_PATH=$CATALINA_HOME/bin
cd $CATALINA_HOME/bin

And then restart Tomcat server.

JavaCC

Posted on April 23, 2008 by Zhou Renjian

I learned to use JavaCC these days. And I found that JavaCC is a great toolkit.

What is JavaCC?

JavaCC is short for Java Compiler Compiler. It is a parser or scanner generator for Java.

Java Compiler Compiler [tm] (JavaCC [tm]) is the most popular parser generator for use with Java [tm] applications. A parser generator is a tool that reads a grammar specification and converts it to a Java program that can recognize matches to the grammar. In addition to the parser generator itself, JavaCC provides other standard capabilities related to parser generation such as tree building (via a tool called JJTree included with JavaCC), actions, debugging, etc.

You can find a lot of language grammar and AST on its site too. I see that C++ and Java 1.5 grammar are there. JavaScript isn’t there. But I find out that Dojo toolkit’s JSLinker provides an ECMAScript 262 one. That is to say, you can create parser and parse C++, Java or JavaScript sources into your AST for any uses. Lots of great features can be imagined.

I spent about a week learning and using JavaCC. I completed a parser for an IDL language named CAR. I also built an Eclipse editor using the generated AST parser. Bang, all things seems working well: keyword hight-lighting, outline view, error probing, … Great toolkit!

Using JavaCC is quite simple, just download javacc-4.0.zip and unzip it, then following its examples tutorial. I sugguest you to start with JavaGrammars and VTransformer examples, as they are about Java grammar.

I think, later, given opportunity, I will try JavaCC out on other projects. Maybe my Java2Script compiler would be benefited from JavaCC.

P.S. Update:
There is a JavaCC Eclipse Plugin project, you can download the plugin to help you writing your syntax file.

Posted in C++, Java, JavaScript | Tagged Compiler, JavaCC | 2 Comments

Java Puzzlers

Posted on November 8, 2007 by Zhou Renjian

They are really very puzzling. Please check them out:
Advanced Topics in Programming Languages: Java Puzzlers, Episode VI

Posted in Java | 2 Comments

Challenging Myself: Find 1st k-Digits Prime Number

Posted on August 16, 2007 by Zhou Renjian

Here is a task to challenge myself: Find the first k digits prime number. And k would be 16, 32, 64, 256, 1024, 8096, 65536, or even bigger.

And the most challenging thing would be most of key calculations will be performed in JavaScript engine hosted inside browsers. And a distributed algorithm will be designed. Algorithm which is similar to Google’s Map/Reduce will be designed. And Apache Hadoop’s Map/Reduce is also in references.

Any comments about this challenge? Or join me to design the algorithm! Or join the distributed computing game.

Keep tuned for how big the number k would be!

PS: After searching around Internet for existed prime number theories, I find out that k=16 is already a challenge for me! There are algorithms to find prime number in format of 2^p+1, whose digits may be bigger than 1000000. But there are not the 1st k-digits algorithms. According to theories, I should have to find out all prime numbers less than given n. But the count of prime numbers less than n is about n / ln (n), which is huge number already for k=16! So my first challenge would be 16.

Posted in AJAX, Challenge, Java, JavaScript | Comments Off

Google Talk Client’s Java SWT Desktop Copy

Posted on August 15, 2007 by Zhou Renjian

Actually, Google Talk Client’s Java SWT desktop copy is the original source for Google Talk Client’s JavaScript copy.

And now it is available for download. Please take a trial.

Sources of Java2Script Google Talk will be available in the next week. Keep tuned.

Posted in AJAX, Google Talk, Java, Java2Script | 8 Comments

Debug into rt.jar

Posted on August 4, 2007 by Zhou Renjian

If you ever use Eclipse trying to debug into Java’s system APIs, such as java.lang.String or java.math.BigInteger, you may find that even you can step over or step in, or have the correct source file open and the correct line of codes highlighted, you still could not see the local variables’ value and method’s parameters are shown in pattern of “arg0”, “arg1” and so on.

Yes, you are already using JDK instead of JRE. But you still have the above problem. What’s wrong? It’s not a bug of Eclipse. It is all about the rt.jar along with the JDK (1.5 or 1.6). In Sun’s official JDK release, debug symbols are already packed into the rt.jar. But local variables’ debug information is not packed. Sun may consider that it is not OK to includes those local variables’ debugging information. That is the source of problem.

To solve this problem, you may need to upgrade to JDK 1.6. Download DEBUG jar file from
http://download.java.net/jdk6/binaries/
Double click it to install a new JDK. And configure your Eclipse to use the new JDK.

For JDK 1.5, recompiling rt.jar by yourself, or other information, please visit
http://forums.java.net/jive/thread.jspa?threadID=399&tstart=0
for a detailed discussion.

Posted in Debug, Java | Comments Off

Why Java Web Start Sucks

Posted on June 14, 2007 by Zhou Renjian

Just have a long waiting time for downloading the *.jar files, and always it comes to a security dialog notifying that the *.jar file is not signed or not signed by known CA while it requires unlimited local access and remote connection access!

How can I accept “Unlimited local access and remote connection access” for unknown *.jar application! Java Applet or Java Web Start really sucks.

Posted in Java | Comments Off

Category Archives: Java

Use jmap and jhat to Detect Memory Leak

Implementing Gmail-alike Web Mail Client by Java2Script

Connect Yahoo IMAP Server by JavaMail

Implementing Web AIM Client

JavaCC

Java Puzzlers

Challenging Myself: Find 1st k-Digits Prime Number

Google Talk Client’s Java SWT Desktop Copy

Debug into rt.jar

Why Java Web Start Sucks

Recent Posts

Recent Comments

Archives

Categories

Blogroll

Meta