Can I make API calls to get catalog information without the context of a specific user?
What I'm looking to do is use the Netflix API to return a list of movies, the associated actors, and the associated directors. Initially I don't care about a specific user (i.e. what movies they have in their list)... I just want the lists first. Or, said differently, I don't want to have to force users to put their netflix credentials in first, prior to showing the list.
I'm imagining that I could say http://api.netflix.com/catalog/titles? and get back a list of titles (or, the first 25 of them). But doing that gives me 401, not authorized.
So, I thought I could do http://api.netflix.com/titles/index?oauth_token=dutmym66ax5kdj99jd9fj9dj9dj&oauth_token_secret=dkj33ndtb8U&application_name=AppName
You can access non-user REST resources without full OAuth. See the section called "Authentication Example: Accessing Catalog Resources" in the Authentication Overview (http://developer.netflix.com/docs/Security).
OK... I'm close, but need one more piece of help. I assume that the oauth_consumer_key paramter is the same as my "API key"... although that could be made clearer in the documentation.
The piece that I'm stuck on: how do I generate this oauth_signature? Specifically, how do I create a "HMAC-SHA1" hash of my request string. The application that I'm using has the capability to code in jscript and VBScript. Any ideas how to create this hash?
Side note: it strikes me that this is needlessly difficult..
Point taken around the key terminology. We're Mashery's first OAuth service provider so some of the key terminology baked into the portal still needs to be addressed.
With regard to signature generation and complexity, it is probably the most difficult aspect of using the API, yet we're following the OAuth standard so we can't come up with something simpler.
Some help for you:
1) An OAuth JavaScript class library can be found under the OAuth resources under the Resources tab.
2) For help creating sigs directly, see "Example: Creating a Base Signature String" in the Auth overview and this section (http://oauth.net/core/1.0/#signing_process.) of the OAuth spec.
I too do not quite understand how the signature works in the Catalog example. Is it the same type of signature that is used for the regular authenticated user requests, built from GET http://api.netlfix.com ... or is built from something else?
If I can follow up a little (since I agree that oAuth's terminology isn't the best)
The Consumer Key and Shared Secret you got when you signed up are effectively your AppID. You use those for general things like looking through the catalog and stuff that doesn't require extra permissions. The oAuth_token and secret are the extra permissions you need to do things like mess with folks queues, and you only REALLY need to use the oAuth_token for things in the /user branch.
Since your identified by your consumer key and secret, your app could be cut off if said consumer key and secret were to fall into The Wrong Hands™. Thus, while you can be a bit open about your consumer key, you really need to protect your shared secret. (this, obviously, goes double for things like oAuth_tokens and secrets since that's user info.) Think of those like your applications username and password.
So, I'd probably shy away from things like building a pure Javascript app that contains all this info. You could secure things by having a proxy that handles the oauth calls for you (kind of like how you'd build a facebook app). There are ways to roll your own AppID to make sure that only your app is talking to you.
To answer Charlie's question, authentication parameters for Catalog requests without an access token should look just like a request for a request token in the normal OAuth request process. It's similar to a "regular authenticated user requests", but doesn't have the access token and thus uses the developer secret vs. the token secret.
To make matters easier, and to test, I'm going to create the request manually, and put it in a browser, just to see the results -- I can automate it later. I need to append the following things:
So, here is my understanding so far as to how to generate/calculate these items.
oauth_consumer_key: this is my developer "API Key" found at http://developer.netflix.com/apps/mykeys, under the heading "Key".
oauth_signature_method: I have chosen to use a JavaScript library to generate my signature... found at http://oauth.googlecode.com/svn/code/javascript/sha1.js. Inside that library is a function called hex_sha1() which takes some text as an input. This means that my signature method is "HMAC-SHA1"
oauth_timestamp: This is calculated as the number of seconds between 1/1/1970 and now().
oauth_nonce: This is a random integer number that, when combined with the oauth_timestamp, must be unique for each request. You don't care how we generate this number... provided it meets the rules. It could be a count of requests, starting at 1, and ending with the last request.
oauth_signature: Here is where I had the most trouble. Given that I am using the JavaScript library found above... I need to call hex_sha1(sometext); What should "sometext" be? Assuming my developer key is u5yxhmhavqmcbpvcmfb5a3ks then my request would be:
Then I pass this string into the signature HMAC-SHA1 function as follows. The important, and tricky part, is what is the consumer secret... in this case it can be found in the http://developer.netflix.com/apps/mykeys, under the heading "Secret". In this case, this is MY secret. Note, I faked this secret, and all the others in this posting.
var signer = OAuth.SignatureMethod.newMethod("HMAC-SHA1",
{consumerSecret: "djmfj23jds", tokenSecret: null});
var Signature = signer.getSignature(BaseString);
NOW finally, I have the correct signature, which I can can append to my request as such. And THIS works.
What my undstanding is that the oauth_signature should just be the shared secret followed by "&". Is this correct? When I ran this url, I initially got an "Invalid time stamp" message. I tried using a timestamp from (unixtimestamp dot com) and then it gives me "Invalid Signature" What am I doing wrong???
You still need to produce a signature. What you have above is the key used by HMAC-SHA1 to generate a sig from the base signature you'd have to create. The authentication overview doc and OAuth spec describe how to do that.
Thanks. Yes, I realize that now. I just followed QlikTechs instructions and wrote a java program to do this.
I'm happy to announce that the f#$king thing is now working. Do you ever just bang your head for hours on something stupid like this? So frustrating. The netflix documentation really isn't as clear as it should be, but that's the case for just about every tech documentation. I've had a frustrating week! I've been taking a crash course on things like web services, java Spring, Hibernate, Struts and every single tutorial I try won't run as advertised. Each and every one seems to have at least one or more problems that prevent it from working without screwing with. It's really unbelievable. Sorry to vent....
Another note. If you're stuck on this, this tread is helpful:
http://developer.netflix.com/forum/read/27647
It does appear to be the case that the parameters in the URL need to be alphabetical order which is something that should be fixed. I can't think of any reason why the order should matter but it does.
ok, One more question. Do you have any idea why I would be getting inconsistent results? I'm running a catalog search and for the same search term, sometimes it works but about half the time if fails with this:
Server returned HTTP response code: 401 "Invalid Signature"
The URL is getting constructed in the same manner, just with a new Timecode and oauth_timestamp and oauth_nonce. I don't understand what the issue could be. I think it was also sending a 403 response code althrough in the last dozen tests I've done that have failed, it's been 401. Have any ideas?
Hmm... When I've seen results like that, it's usually because some value in the URL wasn't being properly escaped. (e.g. one time I'd have a value like "&oauth_sig=abcd1234" another time I'd have "&oauth_sig=abcd+123") It's best to double check those values to make sure that everything is being properly escaped as "%hex" codes.
Ok, I figured it out. You need to URLEncode the signature after you do the HMAC-SHA1. It was (sometimes) returning a result with invalid characters + / etc...
This is why it was failing, but only some of the time. It would be really great if netflix could just post a simple example for Java, php, C# that just does a simple catalog search. I'm sure a lot of people would find this helpful.
It's very simple, but should illustrate what's going on here. Yes, I cheat and use a library rather than hand roll all the calls. (But the nice thing is that the library is also open source and you can scan it easily enough.)
I am unsure why i get this error as I check my Computers system time and it appears to be set correctly.
Any suggestions? I have been banging my head against the wall with this issue for some time now. I have even gone as far as to set my system time to match that of the time in the response header but still no dice.
Hmm... Invalid timestamp usually indicates a clock skew, but if you set your clock correctly, it should work. Oddly, i even tried it from my server and it worked fine. I did push out a fix to OAuthSimple earlier today to address a few bugs, but those shouldn't have been related to this. There's a bit of "slop" that's allowed for requests (I believe it's something like 5 minutes, but I may be wrong about that), and when I've seen that, it usually means that the clock on the requester is running fast.
You could also check the timestamp in the sample URL against the timestamp on the OAuth test page at:
http://developer.netflix.com/resources/OAuthTest
This is probably something wrong in my OAuthSimple library, so my apologies. If I may, what version of PHP are you running and on what OS?
Can I make API calls to get catalog information without the context of a specific user?
What I'm looking to do is use the Netflix API to return a list of movies, the associated actors, and the associated directors. Initially I don't care about a specific user (i.e. what movies they have in their list)... I just want the lists first. Or, said differently, I don't want to have to force users to put their netflix credentials in first, prior to showing the list.
I'm imagining that I could say http://api.netflix.com/catalog/titles? and get back a list of titles (or, the first 25 of them). But doing that gives me 401, not authorized.
So, I thought I could do http://api.netflix.com/titles/index?oauth_token=dutmym66ax5kdj99jd9fj9dj9dj&oauth_token_secret=dkj33ndtb8U&application_name=AppName
But that doesn't work either.
Thoughts?
Message edited by QlikTech 3 years ago
Michael Hart – 3 years ago
You can access non-user REST resources without full OAuth. See the section called "Authentication Example: Accessing Catalog Resources" in the Authentication Overview (http://developer.netflix.com/docs/Security).
Mike
QlikTech – 3 years ago
OK... I'm close, but need one more piece of help. I assume that the oauth_consumer_key paramter is the same as my "API key"... although that could be made clearer in the documentation.
The piece that I'm stuck on: how do I generate this oauth_signature? Specifically, how do I create a "HMAC-SHA1" hash of my request string. The application that I'm using has the capability to code in jscript and VBScript. Any ideas how to create this hash?
Side note: it strikes me that this is needlessly difficult..
Michael Hart – 3 years ago
Point taken around the key terminology. We're Mashery's first OAuth service provider so some of the key terminology baked into the portal still needs to be addressed.
With regard to signature generation and complexity, it is probably the most difficult aspect of using the API, yet we're following the OAuth standard so we can't come up with something simpler.
Some help for you:
1) An OAuth JavaScript class library can be found under the OAuth resources under the Resources tab.
2) For help creating sigs directly, see "Example: Creating a Base Signature String" in the Auth overview and this section (http://oauth.net/core/1.0/#signing_process.) of the OAuth spec.
QlikTech – 3 years ago
One idea would be to provide a REST based API for generating the signature as a nice add on:
http://api.netflix.com/signature?text="blaaa%blaa" returns hash...
Michael Hart – 3 years ago
That would allow others that happened to have your key generate signatures that strongly authentication them as your app, which would be BAD.
Charlie – 3 years ago
I too do not quite understand how the signature works in the Catalog example. Is it the same type of signature that is used for the regular authenticated user requests, built from GET http://api.netlfix.com ... or is built from something else?
QlikTech – 3 years ago
For anyone else looking for the JavaScript library... here is the link http://oauth.net/code
JR Conlin – 3 years ago
If I can follow up a little (since I agree that oAuth's terminology isn't the best)
The Consumer Key and Shared Secret you got when you signed up are effectively your AppID. You use those for general things like looking through the catalog and stuff that doesn't require extra permissions. The oAuth_token and secret are the extra permissions you need to do things like mess with folks queues, and you only REALLY need to use the oAuth_token for things in the /user branch.
Since your identified by your consumer key and secret, your app could be cut off if said consumer key and secret were to fall into The Wrong Hands™. Thus, while you can be a bit open about your consumer key, you really need to protect your shared secret. (this, obviously, goes double for things like oAuth_tokens and secrets since that's user info.) Think of those like your applications username and password.
So, I'd probably shy away from things like building a pure Javascript app that contains all this info. You could secure things by having a proxy that handles the oauth calls for you (kind of like how you'd build a facebook app). There are ways to roll your own AppID to make sure that only your app is talking to you.
Michael Hart – 3 years ago
To answer Charlie's question, authentication parameters for Catalog requests without an access token should look just like a request for a request token in the normal OAuth request process. It's similar to a "regular authenticated user requests", but doesn't have the access token and thus uses the developer secret vs. the token secret.
QlikTech – 3 years ago
From reading the forum, there seems to be quite a bit of confusion on how to create a proper oauth_signature.
Through much trial and error, I have gotten it to work (I think).
Let me see if I can make the question easier. Assume I want to make the request below:
http://api.netflix.com/catalog/titles/movies/18704531
To make matters easier, and to test, I'm going to create the request manually, and put it in a browser, just to see the results -- I can automate it later. I need to append the following things:
oauth_consumer_key=
oauth_signature_method=
oauth_timestamp=
oauth_nonce=
oauth_signature=
So, here is my understanding so far as to how to generate/calculate these items.
oauth_consumer_key: this is my developer "API Key" found at http://developer.netflix.com/apps/mykeys, under the heading "Key".
oauth_signature_method: I have chosen to use a JavaScript library to generate my signature... found at http://oauth.googlecode.com/svn/code/javascript/sha1.js. Inside that library is a function called hex_sha1() which takes some text as an input. This means that my signature method is "HMAC-SHA1"
oauth_timestamp: This is calculated as the number of seconds between 1/1/1970 and now().
oauth_nonce: This is a random integer number that, when combined with the oauth_timestamp, must be unique for each request. You don't care how we generate this number... provided it meets the rules. It could be a count of requests, starting at 1, and ending with the last request.
oauth_signature: Here is where I had the most trouble. Given that I am using the JavaScript library found above... I need to call hex_sha1(sometext); What should "sometext" be? Assuming my developer key is u5yxhmhavqmcbpvcmfb5a3ks then my request would be:
http://api.netflix.com/catalog/titles/movies/18704531?oauth_consumer_key=u5yxhmhavqmcbpvcmfb5a3ks&
oauth_signature_method=HMAC-SHA1&oauth_timestamp=1221688587&oauth_nonce=1
But I need to append a new parameter called "oauth_signature" to this request.
So... First thing I do is URI encode my request which I do in javascript as follows:
var actual = OAuth.SignatureMethod.getBaseString({method: "GET", action: "http://api.netflix.com/catalog/titles/movies/18704531", parameters: {
oauth_consumer_key: "u5yxhmhavqmcbpvcmfb5a3ks", oauth_timestamp: "1221688587", oauth_nonce: "1"
, oauth_signature_method: "HMAC-SHA1"
}});
And actual ends up being:
GET&http%3A%2F%2Fapi.netflix.com%2Fcatalog%2Ftitles%2Fmovies%2F18704531&oauth_consumer_key%3Du5yxhmhavqmcbpvcmfb5a3ks%26oauth_nonce%3D1%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1221688587
Then I pass this string into the signature HMAC-SHA1 function as follows. The important, and tricky part, is what is the consumer secret... in this case it can be found in the http://developer.netflix.com/apps/mykeys, under the heading "Secret". In this case, this is MY secret. Note, I faked this secret, and all the others in this posting.
var signer = OAuth.SignatureMethod.newMethod("HMAC-SHA1",
{consumerSecret: "djmfj23jds", tokenSecret: null});
var Signature = signer.getSignature(BaseString);
NOW finally, I have the correct signature, which I can can append to my request as such. And THIS works.
http://api.netflix.com/catalog/titles/movies/18704531?oauth_consumer_key=u5yxhmhavqmcbpvcmfb5a3ks&
oauth_signature_method=HMAC-SHA1&oauth_timestamp=1221688587&oauth_nonce=1&oauth_signature=9wdjWazSiJDjd08sX55YvPhd4gw=
mikey – 3 years ago
Nice explanation, QlikTech! This is exactly correct!
=-mikey-=
pawan – 3 years ago
HI QlikTech!
you have given the good details but please Can you tell me it in C# beacuse form me javascript is mess:(
Best Regards
Pawan
Stephen Gower – 3 years ago
Michael, you said: "You can access non-user REST resources without full OAuth."
It is as easy as generating a simple URL? I tried to follow the instructions and this is what I think is required:
http://api.netflix.com/catalog/titles/movies/18704531?oauth_consumer_key=myconsumerkey&oauth_signature_method=HMAC-SHA1&oauth_timestamp=1228782074&oauth_nonce=348277129172003&oauth_signature=mysharedsecret&
What my undstanding is that the oauth_signature should just be the shared secret followed by "&". Is this correct? When I ran this url, I initially got an "Invalid time stamp" message. I tried using a timestamp from (unixtimestamp dot com) and then it gives me "Invalid Signature" What am I doing wrong???
Michael Hart – 3 years ago
You still need to produce a signature. What you have above is the key used by HMAC-SHA1 to generate a sig from the base signature you'd have to create. The authentication overview doc and OAuth spec describe how to do that.
Stephen Gower – 3 years ago
Thanks. Yes, I realize that now. I just followed QlikTechs instructions and wrote a java program to do this.
I'm happy to announce that the f#$king thing is now working. Do you ever just bang your head for hours on something stupid like this? So frustrating. The netflix documentation really isn't as clear as it should be, but that's the case for just about every tech documentation. I've had a frustrating week! I've been taking a crash course on things like web services, java Spring, Hibernate, Struts and every single tutorial I try won't run as advertised. Each and every one seems to have at least one or more problems that prevent it from working without screwing with. It's really unbelievable. Sorry to vent....
Stephen Gower – 3 years ago
Another note. If you're stuck on this, this tread is helpful:
http://developer.netflix.com/forum/read/27647
It does appear to be the case that the parameters in the URL need to be alphabetical order which is something that should be fixed. I can't think of any reason why the order should matter but it does.
Michael Hart – 3 years ago
It matters because that's the only way the client and server can create the same signature string. Different order = different signatures = failure.
Stephen Gower – 3 years ago
ok, One more question. Do you have any idea why I would be getting inconsistent results? I'm running a catalog search and for the same search term, sometimes it works but about half the time if fails with this:
Server returned HTTP response code: 401 "Invalid Signature"
The URL is getting constructed in the same manner, just with a new Timecode and oauth_timestamp and oauth_nonce. I don't understand what the issue could be. I think it was also sending a 403 response code althrough in the last dozen tests I've done that have failed, it's been 401. Have any ideas?
JR Conlin – 3 years ago
Hmm... When I've seen results like that, it's usually because some value in the URL wasn't being properly escaped. (e.g. one time I'd have a value like "&oauth_sig=abcd1234" another time I'd have "&oauth_sig=abcd+123") It's best to double check those values to make sure that everything is being properly escaped as "%hex" codes.
Stephen Gower – 3 years ago
Ok, I figured it out. You need to URLEncode the signature after you do the HMAC-SHA1. It was (sometimes) returning a result with invalid characters + / etc...
This is why it was failing, but only some of the time. It would be really great if netflix could just post a simple example for Java, php, C# that just does a simple catalog search. I'm sure a lot of people would find this helpful.
JR Conlin – 3 years ago
Great idea Stephen!
*poof*
http://developer.netflix.com/page/resources/sample_php
It's very simple, but should illustrate what's going on here. Yes, I cheat and use a library rather than hand roll all the calls. (But the nice thing is that the library is also open source and you can scan it easily enough.)
Landon Key – 3 years ago
I downloaded the code from:
http://developer.netflix.com/page/resources/sample_php
But when I run the application I get an error:
X-Mashery-Error-Code: ERR_401_TIMESTAMP_IS_INVALID
I am unsure why i get this error as I check my Computers system time and it appears to be set correctly.
Any suggestions? I have been banging my head against the wall with this issue for some time now. I have even gone as far as to set my system time to match that of the time in the response header but still no dice.
Michael Hart – 3 years ago
It sounds like you are doing the right things. Any chance you system time has the incorrect time zone set?
JR Conlin – 3 years ago
Hmm... Invalid timestamp usually indicates a clock skew, but if you set your clock correctly, it should work. Oddly, i even tried it from my server and it worked fine. I did push out a fix to OAuthSimple earlier today to address a few bugs, but those shouldn't have been related to this. There's a bit of "slop" that's allowed for requests (I believe it's something like 5 minutes, but I may be wrong about that), and when I've seen that, it usually means that the clock on the requester is running fast.
You could also check the timestamp in the sample URL against the timestamp on the OAuth test page at:
http://developer.netflix.com/resources/OAuthTest
This is probably something wrong in my OAuthSimple library, so my apologies. If I may, what version of PHP are you running and on what OS?