21 exercises in cURL

HTTP magic at your terminal fingertips

 Gepostet am August 30, 2019  |  

The 21 curl exercises by Julia Evans are a compact collection of assignments to practice the usage of cURL.

Here is my take at it:

  1. Request https://httpbin.org:
› curl -X GET 'https://httpbin.org'

<!DOCTYPE html>
<html lang="en">

<head>
    <meta charset="UTF-8">
[...]
  1. Request https://httpbin.org/anything. httpbin.org/anything will look at the request you made, parse it, and echo back to you what you requested. curl’s default is to make a GET request:
› curl -X GET 'https://httpbin.org/anything'

{
  "args": {},
  "data": "",
  "files": {},
  "form": {},
  "headers": {
    "Accept": "*/*",
    "Host": "httpbin.org",
    "User-Agent": "curl/7.54.0"
  },
  "json": null,
  "method": "GET",
  "origin": "94.79.129.22, 94.79.129.22",
  "url": "https://httpbin.org/anything"
}
  • As curl’s default is GET the -X GET could be omitted.
  1. Make a POST request to https://httpbin.org/anything
› curl -X POST 'https://httpbin.org/anything'

{
  "args": {},
  "data": "",
  "files": {},
  "form": {},
  "headers": {
    "Accept": "*/*",
    "Content-Length": "0",
    "Host": "httpbin.org",
    "User-Agent": "curl/7.54.0"
  },
  "json": null,
  "method": "POST",
  "origin": "94.79.129.22, 94.79.129.22",
  "url": "https://httpbin.org/anything"
}
  1. Make a GET request to https://httpbin.org/anything, but this time add some query parameters (set value=panda):
› curl -X GET 'https://httpbin.org/anything' -d 'value=pada'

{
"args": {},
"data": "",
"files": {},
"form": {
"value": "pada"
},
"headers": {
"Accept": "_/_",
"Content-Length": "10",
"Content-Type": "application/x-www-form-urlencoded",
"Host": "httpbin.org",
"User-Agent": "curl/7.54.0"
},
"json": null,
"method": "GET",
"origin": "94.79.129.22, 94.79.129.22",
"url": "https://httpbin.org/anything"
}

1. Request google's robots.txt file (www.google.com/robots.txt):
```zsh
› curl -X GET 'https://www.google.com/robots.txt'

User-agent: \*
Disallow: /search
Allow: /search/about
Allow: /search/static
[...]
  1. Make a GET request to https://httpbin.org/anything and set the header User-Agent: elephant.
› curl -X GET 'https://httpbin.org/anything' -H 'User-Agent: elephant'

{
"args": {},
"data": "",
"files": {},
"form": {},
"headers": {
"Accept": "_/_",
"Host": "httpbin.org",
"User-Agent": "elephant"
},
"json": null,
"method": "GET",
"origin": "94.79.129.22, 94.79.129.22",
"url": "https://httpbin.org/anything"
}
  1. Make a DELETE request to https://httpbin.org/anything
› curl -X DELETE 'https://httpbin.org/anything'

{
"args": {},
"data": "",
"files": {},
"form": {},
"headers": {
"Accept": "_/_",
"Host": "httpbin.org",
"User-Agent": "curl/7.54.0"
},
"json": null,
"method": "DELETE",
"origin": "94.79.129.22, 94.79.129.22",
"url": "https://httpbin.org/anything"
}
  1. Request https://httpbin.org/anything and also get the response headers
› curl -X GET 'https://httpbin.org/anything' -i

HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: \*
Content-Type: application/json
[...]
{
"args": {},
[...]
}
  1. Make a POST request to https://httpbin.com/anything with the JSON body {“value”: “panda”}:
› curl -X POST 'https://httpbin.org/anything' -d '{"value": "panda"}'

{
"args": {},
"data": "",
"files": {},
"form": {
"{\"value\": \"panda\"}": ""
},
"headers": {
"Accept": "_/_",
"Content-Length": "18",
"Content-Type": "application/x-www-form-urlencoded",
"Host": "httpbin.org",
"User-Agent": "curl/7.54.0"
},
"json": null,
"method": "POST",
"origin": "94.79.129.22, 94.79.129.22",
"url": "https://httpbin.org/anything"
}
  • The transmitted data ends up in the "form" field.
  1. Make the same POST request as the previous exercise, but set the Content-Type header to application/json (because POST requests need to have a content type that matches their body). Look at the json field in the response to see the difference from the previous one:
› curl -X POST 'https://httpbin.org/anything' \
    -H 'Content-Type: application/json' \
    -d '{"value": "panda"}'

{
"args": {},
"data": "{\"value\": \"panda\"}",
"files": {},
"form": {},
"headers": {
"Accept": "_/_",
"Content-Length": "18",
"Content-Type": "application/json",
"Host": "httpbin.org",
"User-Agent": "curl/7.54.0"
},
"json": {
"value": "panda"
},
"method": "POST",
"origin": "94.79.129.22, 94.79.129.22",
"url": "https://httpbin.org/anything"
}
  • Nice, the data got explicitly recognised as JSON data.
  1. Make a GET request to https://httpbin.org/anything and set the header Accept-Encoding: gzip (what happens? why?):
› curl -X GET 'https://httpbin.org/anything' -H 'Accept-Encoding: gzip'  
M?A? E????4
??it?´h@?F?.Z??
??ߟ??s$??yI???X?$??lE?B?\*=?c?2??tF?4l?~A??m?ᦵl?{̿b<?]?u??v| ???{?ӊ3?&?I???.9:A??o???r???????????????`??k??Y??,?ڣ?ȿ?)1ja"????DK?K?
@%
  • Looks like whatever got transferred, was ‘gzip’ compressed and got spat out into the shell.
  1. Put a bunch of a JSON in a file and then make a POST request to https://httpbin.org/anything with the JSON in that file as the body:
› curl -X POST 'https://httpbin.org/anything' \
    -H 'Content-Type: application/json' \
    -d @bunchof.json

{
"args": {},
"data": "{ \"firstName\": \"John\", \"lastName\": \"Smith\", \"isAlive\": true, \"age\": 27, \"address\": { \"streetAddress\": \"21 2nd Street\", \"city\": \"New York\", \"state\": \"NY\", \"postalCode\": \"10021-3100\" }, \"phoneNumbers\": [ { \"type\": \"home\", \"number\": \"212 555-1234\" }, { \"type\": \"office\", \"number\": \"646 555-4567\" }, { \"type\": \"mobile\", \"number\": \"123 456-7890\" } ], \"children\": [], \"spouse\": null}",
"files": {},
"form": {},
"headers": {
"Accept": "_/_",
"Content-Length": "447",
"Content-Type": "application/json",
"Host": "httpbin.org",
"User-Agent": "curl/7.54.0"
},
"json": {
"address": {
"city": "New York",
"postalCode": "10021-3100",
"state": "NY",
"streetAddress": "21 2nd Street"
},
"age": 27
[...]
},
"method": "POST",
"origin": "94.79.129.22, 94.79.129.22",
"url": "https://httpbin.org/anything"
}
  1. Make a request to https://httpbin.org/image and set the header ‘Accept: image/png’. Save the output to a PNG file and open the file in an image viewer. Try the same thing with with different Accept: headers:
› curl -X GET 'https://httpbin.org/image' -H 'Accept: image/png' -o someImage.png

% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 8090 100 8090 0 0 18879 0 --:--:-- --:--:-- --:--:-- 18901
  • Nice a piglet
  • Also other image formats are available. Try yourself with curl -X GET 'https://httpbin.org/image'
  1. Make a PUT request to https://httpbin.org/anything :
› curl -X PUT 'https://httpbin.org/anything'

{
"args": {},
"data": "",
"files": {},
"form": {},
"headers": {
"Accept": "_/_",
"Content-Length": "0",
"Host": "httpbin.org",
"User-Agent": "curl/7.54.0"
},
"json": null,
"method": "PUT",
"origin": "94.79.129.22, 94.79.129.22",
"url": "https://httpbin.org/anything"
}
  1. Request https://httpbin.org/image/jpeg, save it to a file, and open that file in your image editor:
› curl -X GET 'https://httpbin.org/image/jpeg' -o someJpeg.jpeg

% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 35588 100 35588 0 0 63869 0 --:--:-- --:--:-- --:--:-- 63777
  • Is that a Black-backed jackal? Black-backed jackal
  1. Request https://www.twitter.com. You’ll get an empty response. Get curl to show you the response headers too, and try to figure out why the response was empty:
› curl -X GET 'https://www.twitter.com' -i

HTTP/2 301
content-length: 0
date: Fri, 30 Aug 2019 16:23:25 GMT
location: https://twitter.com/
server: tsa_o
set-cookie: personalization_id="v1_Q9zuTfFFOE9k7hSxDhPo8g=="; Max-Age=63072000; Expires=Sun, 29 Aug 2021 16:23:25 GMT; Path=/; Domain=.twitter.com
set-cookie: guest_id=v1%3A156718220573814829; Max-Age=63072000; Expires=Sun, 29 Aug 2021 16:23:25 GMT; Path=/; Domain=.twitter.com
strict-transport-security: max-age=631138519
x-connection-hash: d98cf4f8396275a8ba48bedc5451fbc6
x-response-time: 118
  • With content-length: 0 the response is indeed empty. The status message HTTP/2 301 signals that the content was Moved Permanently to location: https://twitter.com/.
  1. Make any request to https://httpbin.org/anything and just set some nonsense headers (like panda: elephant):
› curl -X GET 'https://httpbin.org/anything' -H 'panda: elephant'

{
"args": {},
"data": "",
"files": {},
"form": {},
"headers": {
"Accept": "_/_",
"Host": "httpbin.org",
"Panda": "elephant",
"User-Agent": "curl/7.54.0"
},
"json": null,
"method": "GET",
"origin": "94.79.129.22, 94.79.129.22",
"url": "https://httpbin.org/anything"
}
  • No mayor changes besides that the header gets transferred unaltered, which makes it a nice vector for control- and meta-data.
  1. Request https://httpbin.org/status/404 and https://httpbin.org/status/200. Request them again and get curl to show the response headers:
› curl -X GET 'https://httpbin.org/status/404' -i

HTTP/1.1 404 NOT FOUND
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: \*
[...]

› curl -X GET 'https://httpbin.org/status/200' -i

HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: \*
[...]
  • The responses are almost the same, exept that the HTTP status messages differ, HTTP/1.1 404 NOT FOUND vs. HTTP/1.1 200 OK.
  1. Request https://httpbin.org/anything and set a username and password (with -u username:password)
› curl -X GET 'https://httpbin.org/anything' -u 'username:password' -v
[...]

- Server auth using Basic with user 'username'
  > GET /anything HTTP/1.1
  > Host: httpbin.org
  > Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ=
  > [...]
  > {
  > "args": {},
  > "data": "",
  > "files": {},
  > "form": {},
  > "headers": {
      "Accept": "*/*",
      "Authorization": "Basic dXNlcm5hbWU6cGFzc3dvcmQ=",
      "Host": "httpbin.org",
      "User-Agent": "curl/7.54.0"
  },
  "json": null,
  "method": "GET",
  "origin": "94.79.129.22, 94.79.129.22",
  "url": "https://httpbin.org/anything"
  }
  [...]
  • Most important thing here is see what did cURL do with the -u 'username:password' option, therefor the -v. It transformed it into a Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ= header field. Where the “username:password” got Base64 encoded.
  • And be warned this about as secret as the plaintext of it!
  1. Download the Twitter homepage (https://twitter.com) in Spanish by setting the Accept-Language: es-ES header.
› curl -X GET 'https://twitter.com' -H 'Accept-Language: es-Es'

[...]
  • Could be Spanish…, to much content to parse with my eyes.
  1. Make a request to the Stripe API with curl. (see https://stripe.com/docs/development for how, they give you a test API key). Try making exactly the same request to https://httpbin.org/anything:
#!/bin/sh
curl https://api.stripe.com/v1/payment_intents \
 -u sk_test_Oy**\*\*\***: \
 -d amount=999 \
 -d currency=eur \
 -d payment_method_types[]=card \
 -d receipt_email="jenny.rosen@example.com"

{
"id": "pi_1FD**\*\*\***",
"object": "payment_intent",
"amount": 999,
"amount_capturable": 0,
"amount_received": 0,
"application": null,
"application_fee_amount": null,
"canceled_at": null,
"cancellation_reason": null,
[...]
  • To get the API key you need to do the little sign up dance, but with the examples on their site, this went smoothly.
  • Not so sure about the second part of the exercise, as well as I don’t publish the test API key here, I would not send that API key to any foreign server. So I didn’t.

All in all a nice exercise. Julia Evans, thank you for the inspiration.

[Mehr]
cURL  programming  shell  zsh