Friday, February 15, 2008

Scraping first 10 urls

// Replace this with a developer key from http://developer.yahoo.com
String appid = "YbULhZfV34ERqMXMNQP14Opd68RDsU1n0hhNy_kyqZIKEWGJKYNOa7YgWtfmfvs-";

SearchClient client = new SearchClient(appid);
String query = "processing.org";
WebSearchRequest request = new WebSearchRequest(query);

// (Optional) Set the maximum number of results to download
//request.setResults(30);

try {
WebSearchResults results = client.webSearch(request);
// Print out how many hits were found
println("Displaying " + results.getTotalResultsReturned() +
" out of " + results.getTotalResultsAvailable() + " hits.");
println();
// Get a list of the search results
WebSearchResult[] resultList = results.listResults();
// Loop through the results and print them to the console

for (int i = 0; i < resultList.length; i++) {
// Print out the document title and URL.
println((i + 1) + ".");
println(resultList[i].getTitle());
println(resultList[i].getUrl());
println();
}

// Error handling below, see the documentation of the Yahoo! API for details
}
catch (IOException e) {
println("Error calling Yahoo! Search Service: " + e.toString());
e.printStackTrace();
}
catch (SearchException e) {
println("Error calling Yahoo! Search Service: " + e.toString());
e.printStackTrace();
}



Here are the results:





Displaying 10 out of 10500 hits.

1.
Processing 1.0 (BETA)
http://processing.org/

2.
Mobile Processing
http://mobile.processing.org/

3.
Download \ Processing 1.0 (BETA)
http://processing.org/download/index.html

4.
Learning \ Processing 1.0 (BETA)
http://processing.org/learning/index.html

5.
Processing 1.0 (BETA) >> Examples
http://dev.processing.org/

6.
hardware.processing.org
http://hardware.processing.org/

7.
width \ Language (API) \ Processing 1.0 (BETA)
http://processing.org/reference/width.html

8.
Environment (IDE) \ Processing 1.0 (BETA)
http://processing.org/reference/environment/index.html

9.
Video \ Libraries \ Processing 1.0 (BETA) \ Processing 1.0 (BETA)
http://processing.org/reference/libraries/video/index.html

10.
int \ Language (API) \ Processing 1.0 (BETA)
http://processing.org/reference/int.html

This is created using Yahoo's Search API.
The yahoo_search-2.0.1 jar file needs to be dragged into the sketch folder along with the code so the search class can be found.

No comments: