Using ofxHttpUtils - blocked access?

Hi All
I wonder if anyone can help?
I have been looking at ofxHttpUtils for posting a username and password to a webpage. The webpage is: http://alpha.chromaroma.com/player-session/new

Looking at the source, there are two fields I want to fill from my app and submit.
They are player_session_email and player_session_password:

  
  
<input class="large" id="player_session_email" name="player_session[email]" size="30" type="text" />  
</p>  
<p>  
<label for="player_session_password">Password</label>  
  
<input class="large" id="player_session_password" name="player_session[password]" size="30" type="password" />  
</p>  
  

I have the following code from the example that Chris O’Shea and Arturo use:

  
  
ofxHttpForm form;  
		form.name = "/player_session";  
		form.method = OFX_HTTP_POST;  
		form.action = "[http://alpha.chromaroma.com/player-session/new";](http://alpha.chromaroma.com/player-session/new";)  
		form.addFormField("player_session[email]", userName);  
		form.addFormField("player_session[password]", passWord);  
		ofxHttpUtil.addForm( form );  
  

I use the standard printf("%s\n", response.responseBody.c_str()); in a ofxHttpResponse but don’t get anything there- but not quite sure what I should expect? Something the page generated after successful login?

which I hoped would work OK. When I check the console I get nothing though… My question is, what I should I be doing next in terms of a submit function? Should I get my submit button to redirect to a certain URL? Does the webserver recognise the IP address so that any access to protected pages can happen now that the username and password are submitted? I am a newbie to accessing via http and over the web, forgive my ignorance!

My mission is to be able to get past this login page, and then be able to scrape data from the page, which I am more happy about doing… I just can’t get past this point! Hope someone can help

if you are modifying the original example, then you should have:

  
 ofAddListener(httpUtils.newResponseEvent,this,&testApp::newResponse);  
httpUtils.start();  

in the setup method. to read the response from the server you need to do it in the newResponse method as it’s done in the example and you should receive the same html page that you get when doing the login from a web browser.

anyway the authentication can be tricky, once you do the login depending where are you trying to access they can be using different methods, if they rely on a cookie in your computer, then it’s not going to work and i don’t think that the poco utilities for http support them.

Many thanks Arturo!

I get a HTTP response of 405 : a ‘Method not allowed’ when I try this…
but I wonder if it’s the point at which I am trying to submit and get responses- the procedure in my code might be wrong. Would the 405 be the error you were thinking of re: cookies and Poco?

I am going to try and persevere with this, thanks again!!

sacculi

Mmm… using the code I write, the output from response.responseBody is just the login page itself, not what is behind it. It’s my code and not the login page- I have tried with a variety of pages and all I get is the login page back. It’s a sign of my eternal noobie-ness :slight_smile:

These are bits of code I am using in testApp.mm (it’s iphone):

  
  
setup  
  
action_url = "[http://www.sacculi.co.uk/wp-login.php";](http://www.sacculi.co.uk/wp-login.php";)  
ofAddListener(httpUtils.newResponseEvent,this,&testApp::newResponse);  
httpUtils.start();  
  

and in draw ( also gets userName and passWord as strings from keyboard):

  
  
ofDrawBitmapString(requestStr,20,100);  
		  
ofDrawBitmapString(responseStr,20,120);  
  

in touchDown (when you press a button):

  
  
if (touch.x >= 40 && touch.x <= 80 && touch.y >= 40 && touch.y <= 80)  
		{  
    ofxHttpForm form;  
    form.action = action_url;  
     form.name = "user-login-form";  
    form.method = OFX_HTTP_POST;  
      
    form.addFormField("user_login", userName);  
    form.addFormField("user_pass", passWord);  
      
    httpUtils.addForm(form);  
    requestStr = "message sent: " + ofToString(counter);  
    httpUtils.addForm( form );	  
}  
  

and the newResponse function looks like this:

  
  
void testApp::newResponse(ofxHttpResponse &response) {  
responseStr = ofToString(response.status) + ": " + response.responseBody;  
}  
  

I’ve got the latest ofxHttpUtils from github, so should be fine there.
Is it the form.action URL perhaps? Does this need to be set to something else? I guessed it had to be the login page not the destination page?
Or is the order in which I am doing things, and I somehow don’t draw the responseStr to the screen at the right point?
I realise that cookies might be an issue, but was hopeful to be able to reach one page at a time! The example above is for a wordpress login as a test page…
thanks for helping!
sacculi

Hi Arturo

yes, on digging deeper it does seem to be the cookies as you suggested.
I have been trying to hack the following:
http://stackoverflow.com/questions/7049-…-th-cookies
but without much success, trying to integrate it with the http post example.

Also, there seems to be something about cookies in poco:
http://pocoproject.org/docs/Poco.Net.HTTPRequest.html

but that’s a bit beyond me :slight_smile: :slight_smile:

sacculi

i can try to implement the cookies in the poco addon, can you tell me where are you trying to access?

Hi Arturo

wow that would be amazing!
The site I want to access and get and set cookie for is:
http://alpha.chromaroma.com/player-session/new

Someone picked up a question I posted on stack overflow about get and set for cookies in poco, and wrote this:

* add ofxHttpForm::setCookies() so you can pass cookies into the module and getCookies() so the module can access them
* modify ofxHttpUtils::doPostForm() to pull cookies from the ofxHttpForm and set them on the Poco HTTPRequest
* modify the ofxHttpRequest constructor to pull the cookies from the Poco HTTPResponse and provide a way for your code to get at them

but I having tried a few times I was struggling to implement it…

thanks so much! this will help enormously with my project!

sacculi

i’ve added something similar to ofxHttpUtils, it’s in the repository in github, just call sendCookies on the ofxHttpUtils instance in setup before sending any request and it will automatically get the cookies from the responses and send them in the requests.

haven’t tested so tell me if it gives you any problem

wow thanks Arturo that’s super quick!
Just checking that it’s in the download from http://github.com/arturoc/ofxHttpUtils
I couldn’t see the changes there maybe looking in the wrong place
thanks again
sacculi

sorry the upload to github failed and i didn’t noticed. it’s there now

Hi Arturo
I gave it a try tonight with a couple of sites and I could not make it work I’m afraid.
I called

  
  
httpUtils.sendReceivedCookies();  
  

in setup() and it all links and builds fine but the returned page has the line about logon failing - cookies need to be enabled, in there.
I also tried to call it in the main code before the main post to see if that made a difference but it doesn’t…

Just in case it was me making some mistakes in my passing in details to httpUtils, is this ok?

  
  
form.action = action_url;  
            form.name = "logon";  
            form.method = OFX_HTTP_POST;  
            form.addFormField("username", userName);  
            form.addFormField("password", passWord);  
            //form.addFile("file","ofw-logo.gif");  
            httpUtils.addForm( form );  
  

where form.name is what I can find from viewing the source of the page, and username and password are also what I see on the source page. Are my strings (userName and passWord) ok where they are?
thanks again!
sacculi

it seems you need some more things to be able to login, from the webpage you posted, the fields of the form are:

authenticity_token: this is a key that you will need to get by parsing the html of the form since it changes each time.
player_session[email]
player_session[password]
player_session[remember_me]

so with ofxHttpUtils:

  
  
            form.action = action_url;  
            form.name = "signin";  
            form.method = OFX_HTTP_POST;  
            form.addFormField("authenticity_token", parsed_token);  
            form.addFormField("player_session[email]", email);  
            form.addFormField("player_session[password]", password);  
            form.addFormField("player_session[remember_me]", 0);  

Thanks Arturo! That makes sense now, I can see what I needed.
The web dev team for the site need to make some changes their end, but it all should work fine now, thanks so much for the top quality support, much appreciated.
sacculi

Hi Arturo
the web devs made some changes to the site regarding login, and now I can do my POSTs using the httpUtils - as long as I can do basic HTTP authentication on the page first.

They have described it thus:

  
  
It can be tested on the command line as follows:  
  
curl -i --user nickname:password --data "title=hello" [http://staging.chromaroma.com/account/secret-notes.xml](http://staging.chromaroma.com/account/secret-notes.xml)  
  
Logging in with a browser to the following URL should then show the posted note:  
  
[http://staging.chromaroma.com/account/secret-note](http://staging.chromaroma.com/account/secret-note)  
  

I’ve had a look in httpUtils and can’t see a way of serving up a user name and password as part of something like NSURLRequest in Obj-C.

Is there another way of doing basic HTTP authentication in OF, for example trying to access http://www.pagetutor.com/keeper/mystash-…-stuff.html, where the strings userName and passWord are served up in the URL and then I can use the httpUtils to post stuff?

thanks so much
sacculi

poco supports basic authentication, i’ve just added support for it to ofxHttpUtils. just call:

  
httpUtils.setBasicAuthentication(user,pass);  

before sending any request and it will add the authentication to every request.

Hi Arturo
delighted to say the basic authentication works fine! working like a dream!! Thanks
could I ask a question about the threading and working of http utils?
Say I made a out of connection to the web- will it store and post the file the next time a connection is made with the web?
In the same manner, does the http utils handle multiple form postings on the thread, so that, if I was on the London underground say, I could complete and post three notes and they would be stacked up and posted when I next get a connection?
thanks so much
sacculi

yes the thread works as a queue, everything you post get’s in the queue and the thread will retry till there’s a connection available, the authentication will be stored also and sent with every request, if you want to remove it just set the user and pass to “”

thanks Arturo!
I have been testing the app on the London underground this week and although it does seem to be putting my posts into the thread, when I close the app and relaunch when next web connected, I never get the message on screen that says that the message has successfully reached the web page (based on response codes). I have wondered if this might the closing and opening of the app, that somehow this interrupts and upsets the thread and that some handling of this is needed. I’m not sure.
Just to give an idea of how the code is working:

  
  
posting the form on touchDown:  
//send the form if submit is hit...  
		  
		if (touch.x >= 5 && touch.x <= 95 && touch.y >=445 && touch.y <=475) {  
			  
			httpUtils.setBasicAuthentication(userName, passWord);  
  
			form.action = action_url; // which is [http://staging.chromaroma.com/account/secret-notes.xml](http://staging.chromaroma.com/account/secret-notes.xml)  
			//form.name = "";  
			form.method = OFX_HTTP_POST;  
			form.addFormField("title", mainPost);  
			  
			  
			  
			httpUtils.addForm( form );  
			  
			  
			posted = true;  
  

and the handling in the newResponse function:

  
  
responseStr = ofToString(response.status);	  
	//denial of service if username and password not recognised  
	  
	cout << response.status << "... the response status\n";  
	  
		  
	if (responseStr.compare("403") == 0) {  
		badResponse = true;  
	} else if (responseStr.compare("200") == 0) {  
		badResponse = false;  
		goodResponse = true;  
	} else if (responseStr.compare("500") == 0){  
		badResponse = true;  
	}  
  

and then finally how the response sets the state of the app:

  
  
if (posted){  
				  
				ofPushStyle();  
				ofSetColor(0, 0, 0);  
				futura.drawString("submitted!", 5, 460);  
				ofPopStyle();  
				  
				if (posted && goodResponse){  
				ofPushStyle();  
				ofSetColor(0, 255, 0);  
					futura.drawString("Posted! Press reset note", 10, 390);  
									  
					ofPushStyle();  
					ofSetColor(0, 0, 0);  
					ofNoFill();  
					  
				ofRect(0, 375, 160, 45);  
					ofPopStyle();  
				  
				ofPopStyle();  
				}  
				  
			}  
  

Is there something I could change in ofxThread? I am a bit unfamiliar with threading.
You have been super helpful, sorry to keep throwing problems out there, it’s my inexperience!
sacculi

i don’t think is nothing related with the thread, if you close and app and restart it again the state of it is not stored unless you do it by your self storing things in xml for example. don’t know for sure how this things work in iphone so i could be wrong

the best thing to do will be to store your requests in an vector or something like that and delete them once they’ve been sent, then if you detect the app is going to close store that vector in a xml, when the app starts read that xml to see if there’s any requests that need to be sent.