parse website

Hi.

I would like to be able to parse the content of a website through OF.
But I have no clue how to read the URL content back in to OF.

This is a sample URL:
https://www.videotron.com/api/1.0/internet/usage/wired/123456788.json?lang=en&caller=www.example.de

thx.

at the moment, we haven’t compiled poco with SSL support – but it seems like this would be really helpful as most web services move to https.

it’s on our to-do list for 0071:

https://github.com/openframeworks/openFrameworks/issues/623

some people have been hacking on this, and I know there are versions of poco compiled for SSL support out there.

http://forum.openframeworks.cc/t/https-request-in-of/7457/0

here’s how to do a simple web request, if it’s helpful:

  
  
#pragma once  
  
#include "ofMain.h"  
  
class testApp : public ofBaseApp{  
  
	public:  
		void setup();  
		void update();  
		void draw();  
  
		void keyPressed  (int key);  
		void keyReleased(int key);  
		void mouseMoved(int x, int y );  
		void mouseDragged(int x, int y, int button);  
		void mousePressed(int x, int y, int button);  
		void mouseReleased(int x, int y, int button);  
		void windowResized(int w, int h);  
		void dragEvent(ofDragInfo dragInfo);  
		void gotMessage(ofMessage msg);  
		  
        void urlResponse(ofHttpResponse & response);  
      
  
};  
  
  

  
  
#include "testApp.h"  
  
//--------------------------------------------------------------  
void testApp::setup(){  
    ofRegisterURLNotification(this);  
}  
  
  
void testApp::urlResponse(ofHttpResponse & response){  
	if(response.status==200 ){  
		cout << response.request.name << endl;  
        cout << response.data.getText() << endl;  
	}else{  
		cout << response.status << " " << response.error << endl;  
	}  
}  
  
  
//--------------------------------------------------------------  
void testApp::update(){  
  
}  
  
//--------------------------------------------------------------  
void testApp::draw(){  
  
}  
  
//--------------------------------------------------------------  
void testApp::keyPressed(int key){  
    ofLoadURLAsync("[http://www.cnn.com","load"](http://www.cnn.com","load"));  
}  
  
//--------------------------------------------------------------  
void testApp::keyReleased(int key){  
  
}  
  
//--------------------------------------------------------------  
void testApp::mouseMoved(int x, int y ){  
  
}  
  
//--------------------------------------------------------------  
void testApp::mouseDragged(int x, int y, int button){  
  
}  
  
//--------------------------------------------------------------  
void testApp::mousePressed(int x, int y, int button){  
  
}  
  
//--------------------------------------------------------------  
void testApp::mouseReleased(int x, int y, int button){  
  
}  
  
//--------------------------------------------------------------  
void testApp::windowResized(int w, int h){  
  
}  
  
//--------------------------------------------------------------  
void testApp::gotMessage(ofMessage msg){  
  
}  
  
//--------------------------------------------------------------  
void testApp::dragEvent(ofDragInfo dragInfo){   
  
}  
  

hope that helps -
zach

depending on what type of data you need back Yahoo Pipes used to be useful for this and may be able to be a proxy for SSL pages

http://pipes.yahoo.com/pipes/

Thanks guys.

Zach’s simple sample gives me this error

OF: OF_LOG_ERROR: ofURLFileLoader Malformed message: Invalid HTTP version string

maybe because the url i am working with is https://www

yes – at the moment, https will not work (because poco is not compiled with SSL support).

you can take a look at the links I provided. also, if you could run php on a server, you could just mirror the content from the https source to http – something like this might work?

  
  
<?php  
    $feed_contents = file_get_contents('[https://example.com/feed.rss'](https://example.com/feed.rss'));  
    header('Content-Type: application/rss+xml');  
    echo $feed_contents;  
?>  
  

found that here:

http://stackoverflow.com/questions/531179/using-a-https-resource-in-yahoo-pipes

  • zach