作者iduhc (菠萝面包)
看板java
标题Re: [问题] 网路爬虫遇到javascript
时间Mon Jun 6 17:52:34 2011
感谢 uziel和caty1010大大的帮忙
下面是我参考网路上的资料所写的,不过他好像没有把转向後的网页抓出来
请问哪里有问题？？？
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import org.apache.commons.httpclient.Header;
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.HttpException;
import org.apache.commons.httpclient.HttpStatus;
import org.apache.commons.httpclient.NameValuePair;
import org.apache.commons.httpclient.methods.PostMethod;
public class test
{
public static void main(String args[])
{
String url = "
http://khh.travel/tw/spots/RecSpotList.aspx";
PostMethod postMethod = new PostMethod(url);
// 填入各个表单域的值
NameValuePair[] data =
{
new NameValuePair("__EVENTTARGET", "Next"),
new NameValuePair("__EVENTARGUMEN", ""),
new NameValuePair("__VIEWSTAT","hidden")
};
// 将表单的值放入postMethod中
postMethod.setRequestBody(data);
// 执行postMethod
HttpClient httpClient=new HttpClient();
int statusCode = 0;
try {
statusCode = httpClient.executeMethod(postMethod);
} catch (HttpException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
// HttpClient对于要求接受后继服务的请求，象POST和PUT等不能自动处理转发
// 301或者302
if (statusCode == HttpStatus.SC_MOVED_PERMANENTLY ||
statusCode == HttpStatus.SC_MOVED_TEMPORARILY) {
// 从头中取出转向的位址
Header locationHeader = postMethod.getResponseHeader("location");
String location = null;
if (locationHeader != null) {
location = locationHeader.getValue();
System.out.println("The page was redirected to:" + location);
} else {
System.err.println("Location field value is null.");
}
}
}
}
ps.我看有一些爬虫类的书好像有介绍到Rhino这个工具,来处理javascript,
有其他大大要介绍一下嘛？虽然用上面方法好像就ok了
--
※ 发信站: 批踢踢实业坊(ptt.cc)
◆ From: 140.116.130.25