threaded Http GET and POST using Poco

Here is a class I’ve started to make:
(updated http://addons.openframeworks.cc/project-…-xhttputils )

It isn’t finished, but wanted to share it at this stage to get some feedback.

For now I have commented out GET, but POST is in and works.

If you wanted to post variables from a html page to a php script for example, you would normally use a form or $_POST values. This can be done using the class like this:

  
  
// .h  
ofxHttpUtils web;  
  
// whilst running...  
  
// this adds variables and their values to the class  
web.addFormField("username", "chris");  
web.addFormField("password", "blah");  
  
// this posts them to some online script  
web.postForm("[http://somedomain.com/save.php");](http://somedomain.com/save.php");)  
  
// clear out the form fields for next frame  
webclearFormFields();  
  

I am using this to save animation to a mysql database. For everyframe of video I am saving every blob, for every blob I have to say every x,y point of the contour.

I found this of course hangs the program whilst Poco sends this data, so now the class is in its own thread.

Each cycle of the thread will lock the mutex, send any data if there is some to be sent, the thread will hang whilst it does this, then unlock the thread.

For the functions like addFormField/postForm/clearFormFields should I be locking the mutex as they edit variables within the class that the threaded function accesses?

Please take a look and let me know what you think.

Hi Chris,

Just tried it, very sweet! I think the reason i didnt get the post working was the prepareSubmit function. Didnt know about that. Adding GET support to this Util would be easy i think, just appending the GET form values to the url. Since i dont know too much about threading, i’m not sure how easy this is but, would it be possible to save the result of the request in a string that could be accessed through the ofxHttpUtil class?

edit: Just saw your comment about sending event when result was returned. That’s much smarter i guess…

/cj

Hi chris

Just been looking at this.

You will need to lock postForm addFormField and clearFormField, if not, you can modify the fields or the url of a post that hasn’t been already sent. But then you will have problems again with the application freezing unless you use non blocking locks

I think the best way to go is to have a different class or struct for forms, so you can send as many post petitions as you want without having to lock. The idea is to create a class that contains all the fields information + url. Each time you send a new post, you can store them in a queue and then the threaded function only pops from that queue and send, and the post method only adds to that queue, that way you don’t need locks at all (perhaps one for consulting the size of the queue and writing to it, but that won’t make the app freeze)

I need something like this for a project so I’m going to add this changes, I’ll post when I have them. If you have something else implemented, let me know…

some code…

it has post and request, blocking and unblocking methods and events for new response:

http://65.111.166.199/openframeworks/ofxHttpUtils.zip

for the record, it has also the ofTimestamp glitch mentioned here

http://forum.openframeworks.cc/t/ofxmailutils/1191/7

so just add typedef Poco::Timestamp ofTimestamp; at the top of ofxHttpUtils.h and it works beautifully.

i mention it just in case someone tries this and doesn’t check the forum every 10 minutes as I do…

hey arturo, I’ve been trying to use the enhanced class and have a question. what if I want to get the raw response from the server? as far as I can see with your examples in the event I only get the reason and status of the response (ie: 200 & ok), but can it be used to get some actual data? for example if I want an xml back from the server…

perhaps it’s some http misunderstanding on my part…

If you take a look to the ofxHttpResponse, it has this fields:

  
	  
int status; 				// return code for the response ie: 200 = OK  
string reasonForStatus;		// text explaining the status  
string responseData;		// the actual response  
ofTimestamp timestamp;		// time of the response  

As it says in the comments, responseData is actually the content of the response.

hmmm, yeah, I looked at that and that’s why I wonder if it’s a misunderstanding on my part. In your ofxHttpResponse struct you have

  
  
	ofxHttpResponse(HTTPResponse& pocoResponse){  
		status=pocoResponse.getStatus();  
		timestamp=pocoResponse.getDate();  
		reasonForStatus=pocoResponse.getReasonForStatus(pocoResponse.getStatus());  
		responseData=pocoResponse.getReason();  
	}  
  
  

so responseData is the reason. Which is, as far as the wikipedia goes, in http:

In HTTP/1.0 and since, the first line of the HTTP response is called the status line and includes a numeric status code (such as “404”) and a textual reason phrase (such as “Not Found”)

So I get OK as responseData in, for example, this xml

  
  
<response operation="getProfile" xsi:schemaLocation="[http://com.thepublic.backbone](http://com.thepublic.backbone) profile.xsd">  
<status>  
<returnCode>200</returnCode>  
<message>OK</message>  
</status>  
  
<data>  
  
<profile visitor-id="717" tag-id="000139" nickname="onethreenine" visuallyImpaired="false">  
  
<profile-data-items>  
<name-value name="HeightData" value="15"/>  
<name-value name="MoodData" value="96"/>  
<name-value name="AgeData" value="95"/>  
....................  
......................  
.....  
  

and I’m really interested on the rest of the xml. I was trying to use the poco HTTPResponse.read with no luck, so finally what I’ve done is to create a new Event called newRawResponse which just passes the whole string of the response and then I just parse that.

Was just wondering if I’m going against any http rule or whatever.

ok, I was looking at the output and thought that the body of the response that was being printed was from responseData, actually it was printed in the debug traces in ofxHttpUtils.

I’ve made some changes, now responseData is responseBody and contains the text of the response and theres also a contentType field that has the mime type of the response. The download is the same.

yes that printf stuff gave me some head scratching.

thanks for the changes!

one other glitch I thought I’d mention: on removeListener you have

  
  
responseEvent += Poco::Delegate<ofxHttpEventListener,ofxHttpResponse>(listener,&ofxHttpEventListener::newResponse);  
  

and I believe it should be

  
  
responseEvent -= Poco::Delegate<ofxHttpEventListener,ofxHttpResponse>(listener,&ofxHttpEventListener::newResponse);  
  

yep, the sign.

thanks jesus, it’s corrected now

art, great changes with the que and adding events.

needed to make sure the files had timestamp in like jesus said, plus also basicevent and delegate needed to be included.

Have updated the files here
http://addons.openframeworks.cc/project-…-xhttputils

I have a problem with the way the events work.

I want to use 4 instances of the http utils class that run on their own thread. Each one has a different task & different url, some get some post, 2 are using threads and two just get url.

My app class structure looks something like this (very simplified):
testApp

  • animSaver
  • animLoader
  • settingsLoader
  • bugReporter

Each sub class has an instance of the httpUtils class, and each sub class extends ofxHttpEventListener, has newResponse function and each class registers : ofxHttpEvents.addListener(this);

The problem i have is that the reponse events are called on ALL instances even though only 1 event has occured.

Does that make sense?

I don’t know if this is the best way to do it or not, but I have added a url property to ofxHttpResponse so that in the newResponse function you can at least see where the data came from (same link as above).

what do you think?
is there another way around the event broadcasting problem?

I second your question, I used the form name as you use the url to see where responses where coming from, but I was also wondering if that was a good way.

One thing I’m thinking about your structure (and I’m totally lucubrating here so pardon if I’m way off) is… wouldn’t it be easier/cleaner if you use just a single instance of ofxHttpUtils?

On a superquick thought I think the approach I’d go for is having a sort of wrapper around ofxHttpUtils which knows everything about what’s happening on the http part (urls, where things come from…) and based on that broadcasts specific events to your application. So instead of listening to newResponse everywhere and having to parse what happened you could listen for “animationSaved” “settingsLoaded” and stuff… I did something tangently related recently and might try to code an example if I get some time this weekend

…hope it makes sense.

Possibly, although I am using getUrl twice and post form twice. getUrl isn’t threaded and I’m wondering if it will interfere with the posting. Because I am saving lots of data through the form, it could be that the thread is busy whilst I try to access data through get. At least by having 4 threads I can do many things at one time. Will look into it though.

At the moment this installation has multiple machines running the same software, all connecting to a local server for database access. There is run time settings, so I can change any settings of the app via my web cms and then every machine will update to the new settings within two minutes.

About the threading stuff, I’m almost sure the classes are thread safe, it shouldn’t matter how many calls you do simultaneously, as every variable is local, and each post receive a different instance of the form.

About the events, you can create your own instance of ofxHttpUtils and each of that instances will have it’s own queue and generate it’s own events. You just need to do:

  
ofxHttpUtils http;  
  
  
http.addListener(this);  
http....  

in each class you need to receive different events.

On a superquick thought I think the approach I’d go for is having a sort of wrapper around ofxHttpUtils which knows everything about what’s happening on the http part (urls, where things come from…) and based on that broadcasts specific events to your application. So instead of listening to newResponse everywhere and having to parse what happened you could listen for “animationSaved” “settingsLoaded” and stuff… I did something tangently related recently and might try to code an example if I get some time this weekend

Yes, that will be a good way of structuring your app. Also it can be useful if each new form posted had an id, so whenever you make a new post/get, the function returns the new id that then you can compare with that in the form. This will need a lock to be sure you don’t use the same id twice.

[quote author=“arturo”]About the threading stuff, I’m almost sure the classes are thread safe, it shouldn’t matter how many calls you do simultaneously, as every variable is local, and each post receive a different instance of the form.

About the events, you can create your own instance of ofxHttpUtils and each of that instances will have it’s own queue and generate it’s own events. You just need to do:
[/quote]

Yeah I am already doing this on 4 seperate instances of httpUtils.

getUrl though is not part of the threaded function. So when I call getUrl it locks up my application whilst it downloads the content.

Maybe we should make this threaded?

The idea is that you have both options, getUrl / submitForm to do get/post requests non-threaded in case you need it and then addUrl / addForm to do the same in a thread so the application doesn’t lock while you make the request.