Huang Zhimin's home

rubyconfchina
郁闷的一天

导入联系人列表的类库

2009-05-14 22:38:10 +0800

上个月在google code上发布了一个导入联系人列表的类库，http://code.google.com/p/contact-list，获取用户的msn和邮箱联系人列表，支持的邮箱包括hotmail, gmail, yahoo, sohu, sina, 163, 126, tom和yeah。算是之前一段时间的小成果吧。

不过由于这个类库的原理是使用抓取网页来分析联系人列表的，所以会因为邮箱网页的改版而无法正确获取联系人列表。 在写代码的时候就碰到过Hotmail改版的情况，所以这个类库是需要不断改版的。还好在发布之前特意写好了测试脚本，很容易找到哪个邮箱出问题了。

今天收到一个用户的email，说他的新浪和搜狐好像无法导入，我回来test了一下，发现新浪和搜狐没问题，倒是yahoo出问题了。于是给他回信，让他把log4j的level设置成debug，然后把调试信息发给我。现在有了用户的反馈，有点小压力，要好好维护好这个类库

Posted in: 专业
28 Comments
  1. zzg at 2009-05-15 17:25:21 +0800 Said,

    测试126 Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/httpclient/HttpMethod at com.huangzhimin.contacts.ContactsImporterFactory.getHotmailContacts(ContactsImporterFactory.java:33) at Test.getCon(Test.java:16) at Test.main(Test.java:26)

  2. Huang Zhimin at 2009-05-16 00:05:21 +0800 Said,

    你下载的是contactlist-1.1.jar吧，这个是不包含依赖类库的，你可以下载contactlist-1.1-jars-with-dependencies.zip，这个包括所有的依赖类库，文档和代码的

  3. zzg at 2009-05-16 09:06:20 +0800 Said,

    com.huangzhimin.contacts.exception.ContactsException: Hotmail protocol has changed at com.huangzhimin.contacts.email.EmailImporter.getContacts(EmailImporter.java:104) at Test.getCon(Test.java:17) at Test.main(Test.java:26) Caused by: com.huangzhimin.contacts.exception.ContactsException: Hotmail protocol has changed at com.huangzhimin.contacts.email.HotmailImporter.doLogin(HotmailImporter.java:78) at com.huangzhimin.contacts.email.EmailImporter.getContacts(EmailImporter.java:100) ... 2 more Caused by: java.lang.IllegalArgumentException: Invalid uri 'GN1K12 2009.02.20.17.24.50 Live1 Unknown LocVer:0 --> <!-- PreprocessInfo: btsa007:TK1ESRVBLDA071, -- Version: 6,500,12108,0 --> <html><init>(HttpMethodBase.java:222) at org.apache.commons.httpclient.methods.GetMethod.<init>(GetMethod.java:89) at com.huangzhimin.contacts.email.EmailImporter.doGet(EmailImporter.java:222) at com.huangzhimin.contacts.email.EmailImporter.doGet(EmailImporter.java:206) at com.huangzhimin.contacts.email.HotmailImporter.doLogin(HotmailImporter.java:67) ... 3 more

  4. zzg at 2009-05-16 09:42:44 +0800 Said,

    对了 你知识库的源码能提供下载吗？

  5. zzg at 2009-05-16 10:34:51 +0800 Said,

    对了，在用ruby or rails开发时使用你知识库的插件，运行时是正常的，但会在控制台抛出一大串错误信息，错误信息大概是你的主机中的软件放弃了一个已建立的连接，是怎么回事呀？

  6. Huang Zhimin at 2009-05-17 23:22:07 +0800 Said,

    应该是hotmail又更改了网页的结构，我会尽快解决的

  7. guoxuan at 2009-05-23 18:44:54 +0800 Said,

    你好，请教个问题 我想用java程序 模拟一个用户登陆他自己的邮箱，发送邮件给别人，思路应该是什么啊 谢谢

  8. Huang Zhimin at 2009-05-23 19:43:04 +0800 Said,

    你可以先用浏览器登录邮箱，发送邮件，其间使用抓包工具（如：wireshark）或者是firefox的插件LiveHttpHeaders来观察浏览器发送的请求和收到的应答。特别要注意Cookie和Referer的值，以及URL Redirect。 然后使用Apache Commons HttpClient来模拟，即发送Http请求，处理Http应答，注意与之前抓包的结果进行比较。

  9. guoxuan at 2009-05-25 07:29:07 +0800 Said,

    谢谢 ，我试下

  10. guoxuan at 2009-05-25 11:59:17 +0800 Said,

    你好，我发送163邮件时，遇到一个错误：cookie不匹配：<?xml version="1.0" encoding="UTF-8"?> <result> FA_SECURITY <messages> <message>Cookie not matched!</message></messages></result> 这是我的cookie设置： this.client.getState().addCookies(new Cookie[] { new Cookie(".163.com", "_ntes_nnid", "e70457cf2650811883494823e4f11111,0", "/",d , false), new Cookie(".163.com", "_ntes_nnid", "e70457cf2650811883494823e4f11111", "/",d , false), new Cookie(".163.com", "City", "0311", "/",d , false), new Cookie(".163.com", "Coremail", "VTxdlOCdoEjGc%PAmWfpuuJRARvkPirLuujcaEWsfXeejL", "/",d , false), new Cookie(".163.com", "logType", "-1", "/",d , false), new Cookie(".163.com", "MAIL163_SSN", "juuyuan001", "/",d , false), new Cookie(".163.com", "ntes_mail_firstpage", "normal", "/",d , false), new Cookie(".163.com", "mail_style", "js3", "/",d , false), new Cookie(".163.com", "mail_uid", sendUrl, "/",d , false), new Cookie(".163.com", "NTES_SESS", "77vPZwqw7enuVQOAD2wjhemENOtbTjuJrnI.sl_hKTZMpvFRpWZ9_iAv7FC1FF3.F5G9OSPozjgeQtjADRW_glDV8_Sua1HF7yJISSfD7V1VN", "/",d , false), new Cookie(".163.com", "NTES_UFC", "1000000000010000000000000000000000000000000000000000000000000000", "/",d , false), new Cookie(".163.com", "P_INFO", sendUrl+"|1243215439|0|mail163|11&19&1243215439836", "/",d , false), new Cookie(".163.com", "S_INFO", "1243215439|0", "/",d , false), new Cookie(".163.com", "USERTRACK", "121.28.92.5.1243210789552014", "/",d , false), new Cookie(".163.com", "wmsvr_domain", "cg2a13.mail.163.com", "/",d , false), });

  11. guoxuan at 2009-05-25 12:00:22 +0800 Said,

    我用HttpWatch查看的： POST /a/s?sid=PAmWfpuuJRARvkPirLuujcaEWsfXeejL&func=mbox:compose HTTP/1.1 Host: cg2a13.mail.163.com User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; zh-CN; rv:1.8.1.20) Gecko/20081217 Firefox/2.0.0.20 Accept: text/javascript Accept-Language: zh-cn,zh;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: gb2312,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Connection: close Content-Type: application/xml Referer: http://cg2a13.mail.163.com/a/j/js3/index.jsp?sid=PAmWfpuuJRARvkPirLuujcaEWsfXeejL Content-Length: 649 Cookie: Province=0310; City=0311; ntes_mail_firstpage=normal; logType=-1; nts_mail_user=juuyuan001:-1:1; P_INFO=juuyuan001@163.com|1243215439|0|mail163|11&19&1243215439836; Coremail=VTxdlOCdoEjGc%PAmWfpuuJRARvkPirLuujcaEWsfXeejL; USERTRACK=121.28.92.5.1243210789552014; mail_style=js3; mail_uid=juuyuan001@163.com; wmsvr_domain=cg2a13.mail.163.com; MAIL163_SSN=juuyuan001; _ntes_nnid=e70457cf2650811883494823e4f11111,0; _ntes_nuid=e70457cf2650811883494823e4f11111; ntes_ucc=; NTES_UFC=1000000000010000000000000000000000000000000000000000000000000000; NTES_SESS=77vPZwqw7enuVQOAD2wjhemENOtbTjuJrnI.sl_hKTZMpvFRpWZ9_iAv7FC1FF3.F5G9OSPozjgeQtjADRW_glDV8_Sua1HF7yJISSfD7V1VN; S_INFO=1243215439|0; oulink_h= Pragma: no-cache Cache-Control: no-cache <?xml version="1.0"?><object><string>deliver</string><object><string>"juuyuan001" <juuyuan001@163.com></string><array><string>"xuan_158737" <xuan_158737@sina.com></string></array><array /><array /><string>444</string><string>444444444444444<div><br><br></div></string><boolean>true</boolean><int>3</int><boolean>true</boolean><boolean>false</boolean><string>gbk</string></object><int>1</int></object>

  12. Huang Zhimin at 2009-05-26 20:00:59 +0800 Said,

    一般你不需要手动设置cookie的，定义cookie的策略为client.getParams().setCookiePolicy(CookiePolicy.BROWSER_COMPATIBILITY);即可，HttpClient会帮你自动处理Cookie值的

  13. qqq at 2009-07-01 23:36:33 +0800 Said,

    为什么没有qq邮箱的呢

  14. Huang Zhimin at 2009-07-02 20:33:59 +0800 Said,

    因为qq邮箱登录时需要输验证码，所以没法模拟

  15. qqq at 2009-07-03 21:41:22 +0800 Said,

    http://www.yue360.com/grabbers.001/grabbers.html 你看这个网站，他就可以导入qq好友来

  16. Huang Zhimin at 2009-07-03 22:25:58 +0800 Said,

    这个网站qq邮箱也是特别处理，需要输入验证码。网页上当然可以这么做，也就是把qq邮箱登录页面的验证码拿过来让用户输入。我做的是类库，希望给用户提供的是自动处理，而不是还要手动输入，所以对于qq这种需要输入验证码的另类邮箱，我是不想去支持的。另外，我对于在网站上输入邮箱、密码是不会尝试的，万一对方是“钓鱼”网站怎么办呢？

  17. wgl at 2009-07-07 16:12:40 +0800 Said,

    com.huangzhimin.contacts.exception.ContactsException: 126 protocol has changed at com.huangzhimin.contacts.email.EmailImporter.getContacts(EmailImporter.java:104) at com.testEmail.Email.main(Email.java:51) Caused by: com.huangzhimin.contacts.exception.ContactsException: 126 protocol has changed at com.huangzhimin.contacts.email.OneTwoSixImporter.parseContacts(OneTwoSixImporter.java:124) at com.huangzhimin.contacts.email.EmailImporter.getContacts(EmailImporter.java:101) ... 1 more Caused by: java.net.ConnectException: Connection timed out: connect at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333) at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195) at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366) at java.net.Socket.connect(Socket.java:507) at java.net.Socket.connect(Socket.java:457) at java.net.Socket.<init>(Socket.java:365) at java.net.Socket.<init>(Socket.java:238) at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:80) at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:122) at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) at com.huangzhimin.contacts.email.EmailImporter.doGet(EmailImporter.java:226) at com.huangzhimin.contacts.email.EmailImporter.doGet(EmailImporter.java:206) at com.huangzhimin.contacts.email.OneTwoSixImporter.parseContacts(OneTwoSixImporter.java:101)

  18. Huang Zhimin at 2009-07-07 18:46:23 +0800 Said,

    Connection timed out，是你的网络有点问题，你先确认下能不能通过浏览器访问126邮箱

  19. diff at 2009-07-22 14:58:53 +0800 Said,

    java.lang.ArrayIndexOutOfBoundsException: 1 at com.huangzhimin.contacts.utils.UnicodeChinese.<clinit>(UnicodeChinese.java:34) at com.huangzhimin.contacts.email.HotmailImporter.parseContacts(HotmailImporter.java:101) at com.huangzhimin.contacts.email.EmailImporter.getContacts(EmailImporter.java:99) at com.xsoft.util.contact.http.HotmailHandler.getContactList(HotmailHandler.java:17) at com.xsoft.util.contact.http.HotmailHandler.main(HotmailHandler.java:34) 哥们，谢谢你的工具， 我访问hotmail.com出错，是否看看是网站更新的原因，还是我的问题

  20. Huang Zhimin at 2009-07-22 20:01:54 +0800 Said,

    正在解决这个问题，好像只有在windows下面才有这样的错误。 不过我这里没有windows环境，争取这周修复，发布1.4版本

  21. Huang Zhimin at 2009-07-25 21:54:48 +0800 Said,

    @diff 已经修复，请看http://huangzhimin.com/projects/1-contact-list中关于Encoding的部分

  22. diff at 2009-07-26 10:48:16 +0800 Said,

    谢谢哥们,再此先谢过

  23. abc at 2009-08-13 14:34:19 +0800 Said,

    com.huangzhimin.contacts.exception.ContactsException: Hotmail protocol has changed at com.huangzhimin.contacts.email.EmailImporter.getContacts(EmailImporter.java:102) at com.msn.ContactListTest.main(ContactListTest.java:45) Caused by: com.huangzhimin.contacts.exception.ContactsException: Hotmail protocol has changed at com.huangzhimin.contacts.email.HotmailImporter.doLogin(HotmailImporter.java:80) at com.huangzhimin.contacts.email.EmailImporter.getContacts(EmailImporter.java:98) ... 1 more Caused by: com.huangzhimin.contacts.exception.ContactsException: Can't find from url at com.huangzhimin.contacts.email.EmailImporter.getFormUrl(EmailImporter.java:482) at com.huangzhimin.contacts.email.HotmailImporter.doLogin(HotmailImporter.java:56) ... 2 more

  24. siemen95 at 2009-09-04 15:27:46 +0800 Said,

    您好，请看看下面这个异常是什么问题，我用的是JDK5： Exception in thread "main" java.lang.UnsupportedClassVersionError: Bad version number in .class file at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:620) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:124) at java.net.URLClassLoader.defineClass(URLClassLoader.java:260) at java.net.URLClassLoader.access$100(URLClassLoader.java:56) at java.net.URLClassLoader$1.run(URLClassLoader.java:195) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:188) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268) at java.lang.ClassLoader.loadClass(ClassLoader.java:251) at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319) at com.huangzhimin.contacts.email.EmailImporter.parseJSON(EmailImporter.java:451) at com.huangzhimin.contacts.email.HotmailImporter.parseContacts(HotmailImporter.java:97) at com.huangzhimin.contacts.email.EmailImporter.getContacts(EmailImporter.java:99)

  25. Huang Zhimin at 2009-09-05 19:52:11 +0800 Said,

    应该是JDK版本的问题，我是用1.6开发的，你可以试试直接编译源码试试

  26. 呵呵 at 2010-03-02 13:11:49 +0800 Said,

    大侠，有事请教，如果用户输入的是错误账号或者密码返回的错误（如下），我向提示用户密码错误，但是jar包里面采用异步方式。无法捕获啊，而且，就是算密码输错，上次输入得到的邮件联系人的还是存在啊。怎么办啊 rath.msnm.NotificationProcessor: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at rath.msnm.AbstractProcessor.run(AbstractProcessor.java:373) Caused by: java.io.IOException: Server returned HTTP response code: 401 for URL: https://loginnet.passport.com/login2.srf at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1305) at com.sun.net.ssl.internal.www.protocol.https.HttpsURLConnectionOldImpl.getInputStream(HttpsURLConnectionOldImpl.java:204) at rath.msnm.util.TWN.getTNPImpl(TWN.java:121) at rath.msnm.util.TWN.getTNP(TWN.java:90) at rath.msnm.NotificationProcessor.processAuth(NotificationProcessor.java:360) ... 5 more

  27. Huang Zhimin at 2010-03-02 16:19:54 +0800 Said,

    下个版本我会去掉msn的jar包，采用web service的方式获取msn的联系人，应该就可以解决你这个问题了

  28. 求救 at 2010-04-20 09:43:12 +0800 Said,

    我出现和26一样的问题，继续关注...

Add a Comment

2d33d8fb1837759038ee07320b400eb46eae5fdb