Lines Matching full:to

14 .\" * You may opt to use, copy, modify, merge, publish, distribute and/or sell
15 .\" * copies of the Software, and permit persons to whom the Software is
16 .\" * furnished to do so, under the terms of the COPYING file.
27 This document attempts to describe the general principles and some basic
28 approaches to consider when programming with libcurl. The text will focus
32 This document will refer to 'the user' as the person writing the source code
34 What will be generally referred to as 'the program' will be the collected
39 refer to their respective man pages.
42 There are many different ways to build C programs. This chapter will assume a
44 read this to get general information that may apply to your environment as
47 Your compiler needs to know where the libcurl headers are located. Therefore
48 you must set your compiler's include path to point to the directory where you
49 installed them. The 'curl-config'[3] tool can be used to get this information:
54 When having compiled the program, you need to link your object files to create
55 a single executable. For that to succeed, you need to link with libcurl and
58 command line. To figure out which flags to use, once again the 'curl-config'
59 tool comes to the rescue:
67 properly at build-time, libcurl will be built with SSL support. To figure out
73 And if SSL is supported, the keyword 'SSL' will be written to stdout,
79 When you write your configure script to detect libcurl and setup variables
85 The people behind libcurl have put a considerable effort to make libcurl work
89 are only very few minor considerations that differ. If you just make sure to
95 means it should be done exactly once, no matter how many times you intend to
100 and it takes one parameter which is a bit pattern that tells libcurl what to
111 should not tell libcurl to do this as well.
115 application. This only needs to be done once for each application so if your
127 then do the reversed operations to cleanup the resources the
130 Repeated calls to \fIcurl_global_init(3)\fP and \fIcurl_global_cleanup(3)\fP
134 It is considered best-practice to determine libcurl features at run-time
147 interface is detailed in a separate chapter further down. You still need to
152 need one handle for each easy session you want to perform. Basically, you
153 should use one handle for every thread you plan to use for transferring. You
160 It returns an easy handle. Using that you proceed to the next step: setting
166 transfers will be made. Options remain set in the handle until set again to
170 If you at any point would like to blank all previously set options for a
175 Many of the options you set in libcurl are "strings", pointers to data
180 One of the most basic properties to set in the handle is the URL. You set your
181 preferred URL to transfer with \fICURLOPT_URL(3)\fP in a manner similar to:
187 Let's assume for a while that you want to receive data as the URL identifies a
188 remote resource you want to get here. Since you write a sort of application
189 that needs this transfer, I assume that you would like to get the data passed
190 to you directly instead of simply getting it passed to stdout. So, you write
195 You tell libcurl to pass all data to this function by issuing a function
196 similar to this:
211 will then simply output the received data to stdout. You can have the default
212 callback write the data to a different file handle by passing a 'FILE *' to a
215 Now, we need to take a step back and have a deep breath. Here's one of those
217 libcurl won't be able to operate on files opened by the program. Thus, if you
219 \fICURLOPT_WRITEDATA(3)\fP, it will crash. You should therefore avoid this to
229 There are of course many more options you can set, and we'll get back to a few
230 of them later. Let's instead continue to the actual transfer:
234 \fIcurl_easy_perform(3)\fP will connect to the remote site, do the necessary
240 passed to it, libcurl will abort the operation and return with an error code.
244 you, you can use the \fICURLOPT_ERRORBUFFER(3)\fP to point libcurl to a buffer
247 If you then want to transfer another file, the handle is ready to be used
249 you intend to make another transfer. libcurl will then attempt to re-use the
255 complication for you. Given simply the URL to a file, libcurl will take care
256 of all the details needed to get the file moved from one machine to another.
259 libcurl is thread safe but there are a few exceptions. Refer to
269 \fICURLOPT_VERBOSE(3)\fP option to 1. It'll cause the library to spew out the
272 adding the headers in the received output to study is also a clever way to get
276 Of course, there are bugs left. We need to know about them to be able to fix
287 and if you're trying to do funny things, you might very well understand
288 libcurl and how to use it better if you study the appropriate RFC documents
291 .SH "Upload Data to a Remote Site"
292 libcurl tries to keep a protocol independent approach to most transfers, thus
293 uploading to a remote FTP site is very similar to uploading data to a HTTP
297 one. Then you set the URL to operate on just like before. This is the remote
300 Since we write an application, we most likely want libcurl to get the upload
301 data by asking us for it. To make it do that, we set the read callback and
302 the custom pointer libcurl will pass to our read callback. The read callback
303 should have a prototype similar to:
307 Where bufptr is the pointer to a buffer we fill in with data to upload and
309 of data we can return to libcurl in this call. The 'userp' pointer is the
310 custom pointer we set to point to a struct of ours to pass private data
317 Tell libcurl that we want to upload:
332 supplied callback to get the data to upload. The program should return as much
333 data as possible in every invoke, as that is likely to make the upload perform
339 to be able to download or upload the data of your choice. libcurl offers
340 several ways to specify them.
351 libcurl also provides options to set various passwords. The user name and
353 \fICURLOPT_USERPWD(3)\fP option. The argument passed to libcurl should be a
354 char * to a string in the format "user:password". In a manner like this:
359 users who need to authenticate themselves to a proxy they use. libcurl offers
361 similar to the \fICURLOPT_USERPWD(3)\fP option like this:
369 ability to use this file to figure out what set of user name and password to
370 use for a particular host. As an extension to the normal functionality,
371 libcurl also supports this file for non-FTP protocols such as HTTP. To make
385 at least you could leave it out and have libcurl attempt to do its job
389 To pass the known private key password to libcurl:
394 The previous chapter showed how to set user name and password for getting
396 many different ways a client can provide those credentials to the server and
397 you can control which way libcurl will (attempt to) use them. The default HTTP
401 At the time of this writing, libcurl can be built to use: Basic, Digest, NTLM,
402 Negotiate (SPNEGO). You can tell libcurl which one to use
407 And when you send authentication to a proxy, you can also set authentication
412 Both these options allow you to set multiple types (by ORing them together),
414 claims to support. This method does however add a round-trip since libcurl
421 with specific types) which allows libcurl to use whatever method it wants.
427 We get many questions regarding how to issue HTTP POSTs with libcurl the
432 pages using the <form> tag uses. We provide a pointer to the data and tell
433 libcurl to post it all to the remote site:
444 \fICURLOPT_POSTFIELDS(3)\fP, this automatically switches the handle to use
447 Ok, so what if you want to post binary data that also requires you to set the
449 being able to do strlen() on the data to figure out the size, so therefore we
452 then passing that list to libcurl.
474 formposts were introduced as a better way to post (possibly large) binary data
480 to libcurl. To make that easier, libcurl provides \fIcurl_formadd(3)\fP. Using
481 this function, you add parts to the form. When you're done adding parts, you
511 that describe the individual content-type, size etc. To enable your
512 application to handicraft this formpost even more, libcurl allows you to
513 supply your own set of custom headers to such an individual form part. You can
514 of course supply headers to as many parts as you like, but this little example
515 will show how you set headers to one specific part when you add that to the
535 changed even if you do call \fIcurl_easy_perform(3)\fP, you may need to tell
536 curl to go back to a plain GET request if you intend to do one as your next
537 request. You force an easyhandle to go back to GET by using the
542 Just setting \fICURLOPT_POSTFIELDS(3)\fP to "" or NULL will *not* stop libcurl
543 from doing a POST. It will just make it POST without any data to send!
552 \fICURLOPT_NOPROGRESS(3)\fP to zero. This option is set to 1 by default.
555 what instead is interesting is the ability to specify a progress
556 callback. The function pointer you pass to libcurl will then be called on
560 a pointer to a function that matches this prototype:
571 argument, the 'clientp' is the pointer you pass to libcurl with
576 There's basically only one thing to keep in mind when using C++ instead of C
595 What "proxy" means according to Merriam-Webster: "a person authorized to act
600 access to employees through their proxies. Network clients or user-agents ask
605 will ask the proxy for it instead of trying to connect to the actual host
613 HTTP URL will be still be passed to the HTTP proxy to deliver back to
614 libcurl. This happens transparently, and an application may not need to
615 know. I say "may", because at times it is very important to understand that
622 To tell libcurl to use a proxy at a given port number:
627 pass that information similar to this:
631 If you want to, you can specify the host name only in the
636 it will default to assume a HTTP proxy):
642 libcurl automatically checks and uses a set of environment variables to know
643 what proxies to use for certain protocols. The names of the variables are
646 name of a proxy to use when the input URL is HTTP. Following the same rule,
649 different HTTP proxies to be used.
656 and that is most likely *not* the one you would like it to be.
665 variables, set the proxy name to "" - an empty string - with
669 SSL is for secure point-to-point connections. This involves strong encryption
670 and similar things, which effectively makes it impossible for a proxy to
672 discussed. Instead, the only way to have SSL work over a HTTP proxy is to ask
673 the proxy to tunnel trough everything without being able to check or fiddle
677 proxy for a straight connection to the target host on a specified port. This
678 is made with the HTTP request CONNECT. ("please mr proxy, connect me to that
684 organizations prevent this kind of tunneling to other destination port numbers
688 As explained above, tunneling is required for SSL to work and often even
689 restricted to the operation intended for SSL; HTTPS.
691 This is however not the only time proxy-tunneling might offer benefits to
694 As tunneling opens a direct connection from your application to the remote
695 machine, it suddenly also re-introduces the ability to do non-HTTP
702 Tell libcurl to use proxy tunneling like this:
706 In fact, there might even be times when you want to do plain HTTP
707 operations using a tunnel like this, as it then enables you to operate on
708 the remote server instead of asking the proxy to do so. libcurl will not
715 requested URL as input, returns information to the browser on how to connect
717 should be used), "PROXY host:port" (to tell the browser where the proxy for
718 this particular URL is) or "SOCKS host:port" (to direct the browser to a SOCKS
721 libcurl has no means to interpret or evaluate Javascript and thus it doesn't
733 - Ask your admins to stop this, for a static proxy setup or similar.
735 .SH "Persistence Is The Way to Happiness"
738 the way to go.
741 connection alive and open. A subsequent request using the same easy handle to
742 the same host might just be able to use the already open connection! This
745 Even if the connection is dropped, all connections involving SSL to the same
751 without permission to login again like on many FTP servers only allowing N
752 persons to be logged in at the same time.
754 libcurl caches DNS name resolving results, to make lookups of a previously
760 Each easy handle will attempt to keep the last few connections alive for a
761 while in case they are to be used again. You can set the size of this "cache"
766 To force your upcoming request to not use an already existing connection (it
767 will even close one first if there happens to be one alive to the same host
768 you're about to operate on), you can do that by setting
769 \fICURLOPT_FRESH_CONNECT(3)\fP to 1. In a similar spirit, you can also forbid
770 the upcoming request to be "lying" around and possibly get re-used after the
771 request by setting \fICURLOPT_FORBID_REUSE(3)\fP to 1.
774 When you use libcurl to do HTTP requests, it'll pass along a series of headers
775 automatically. It might be good for you to know and understand these. You
780 the name of the server we want to talk to. This includes the port number if
787 When doing POST requests, libcurl sets this header to \&"100-continue" to ask
798 programming you may need to change the traditional HTTP (or FTP or...)
799 manners. You may need to change words, headers or various data.
806 is there for you. It is very simple to use:
813 keyword if you want to. You're the boss.
816 HTTP-like protocols pass a series of headers to the server when doing the
817 request, and you're free to pass any amount of extra headers that you
821  struct curl_slist *headers=NULL; /* init to NULL is important */
835 Accept: or Host: don't contain the data you want them to contain, you can
845 the header from being sent. For instance, if you want to completely prevent the
846 \&"Accept:" header from being sent, you can disable it with code similar to this:
857 when doing a non-GET HTTP operation, libcurl will switch over to "chunked"
858 upload, even though the size of the data to upload might be known. By default,
859 libcurl usually switches over to chunked upload automatically if the upload
864 All HTTP requests includes the version number to tell the server which version
867 you can tell libcurl to use 1.0 instead by doing something like this:
874 you want to make, for example, your FTP transfers to behave differently.
876 Sending custom commands to a FTP server means that you need to send the
880 a data-connection must be left to libcurl's own judgement. Also be aware
881 that libcurl will do its very best to change directory to the target
883 or similar) you might confuse libcurl and then it might not attempt to
889  headers = curl_slist_append(headers, "DELE file-to-remove");
891  /* pass the list of custom commands to the handle */
899 If you would instead want this operation (or chain of operations) to happen
900 _after_ the data transfer took place the option to \fIcurl_easy_setopt(3)\fP
904 The custom FTP command will be issued to the server in the same order they are
905 added to the list, and if a command gets an error code returned back from the
907 error code (CURLE_QUOTE_ERROR). Note that if you use \fICURLOPT_QUOTE(3)\fP to
911 If you set the \fICURLOPT_HEADER(3)\fP to 1, you will tell libcurl to get
915 The option to enable headers or to run custom FTP commands may be useful to
920 If you do want to list the contents of a FTP directory using your own defined
922 default one for listing directories but you're free to pass in your idea of a
927 the name and value to the client, and expects it to get sent back on every
928 subsequent request to the server that matches the particular conditions
932 In real-world cases, servers send new cookies to replace existing ones to
933 update them. Server use cookies to "track" users and to keep "sessions".
935 Cookies are sent from server to clients with the header Set-Cookie: and
936 they're sent from clients to servers with the Cookie: header.
938 To just send whatever cookie you want to a server, you can use
939 \fICURLOPT_COOKIE(3)\fP to set a cookie string like this:
943 In many cases, that is not enough. You might want to dynamically save
944 whatever cookies the remote server passes to you, and make sure those cookies
947 One way to do this, is to save all headers you receive in a plain file and
948 when you make a request, you tell libcurl to read the previous headers to
949 figure out which cookies to use. Set the header file to read cookies from with
957 is used. Many times this is enough, and you may not have to save the cookies
958 to disk at all. Note that the file you specify to \fICURLOPT_COOKIEFILE(3)\fP
959 doesn't have to exist to enable the parser, so a common way to just enable the
960 parser and not read any cookies is to use the name of a file you know doesn't
973 enables cookies to get passed on properly between multiple handles without any
980 back to haunt you. libcurl offers several different ways to customize how the
983 libcurl can either connect to the server a second time or tell the server to
984 connect back to it. The first option is the default and it is also what works
986 libcurl then tells the server to open up a new port and wait for a second
988 work it tries PASV instead. (EPSV is an extension to the original FTP spec
992 \fICURLOPT_FTP_USE_EPSV(3)\fP to zero.
994 In some cases, you will prefer to have the server connect back to you for the
997 informs the remote server which IP address and port number to connect to.
998 This is made with the \fICURLOPT_FTPPORT(3)\fP option. If you set it to "-",
999 libcurl will use your system's "default IP address". If you want to use a
1000 particular IP, you can set the full IP address, a host name to resolve to an
1004 When doing the "PORT" approach, libcurl will attempt to use the EPRT and the
1006 this behavior by setting \fICURLOPT_FTP_USE_EPRT(3)\fP to zero.
1015 What might be even more useful, is libcurl's ability to separate the headers
1017 different pointer to pass to the ordinary write callback by setting
1020 Or, you can set an entirely separate function to receive the headers, by using
1023 The headers are passed to the callback function one by one, and you can
1024 depend on that fact. It makes it easier for you to add custom header parsers
1037 caution and precautions are taken to mitigate many kinds of risks encountered
1039 powerful library, however, which allows application writers to make trade offs
1040 between ease of writing and exposure to potential risky operations.  If
1041 used the right way, you can use libcurl to transfer data pretty safely.
1055 options to the tool on the command line those options can very likely get read
1056 by other users of your system when they use 'ps' or other tools to list
1059 To avoid this problem, never feed sensitive things to programs using command
1060 line options. Write them to a protected file and use the \-K option to
1064 \&.netrc is a pretty handy file/feature that allows you to login quickly and
1065 automatically to frequently visited sites. The file contains passwords in
1077 anyone on your network or a network nearby yours to just fire up a network
1088 redirects sent by a remote server.  These redirects can refer to any kind of
1093 A redirect to a file: URL would cause the libcurl to read (or write) arbitrary
1094 files from the local filesystem.  If the application returns the data back to
1096 leverage this to read otherwise forbidden data (e.g.
1111 used to mitigate against this kind of attack.
1127 can change the address of the host to a local, private address which a
1129 http://fuzzybunnies.example.com/ could actually resolve to the IP address of a
1134 All the malicious scenarios regarding redirected URLs apply just as well to
1135 non-redirected URLs, if the user is allowed to specify an arbitrary URL that
1136 could point to a private resource. For example, a web app providing a
1142 A malicious FTP server could in response to the PASV command return an IP
1143 address and port number for a server local to the app running libcurl but
1152 would all bypass a naive filter and could allow access to undesired local
1156 organization or server may also be configured to limit IPv4 connections but
1158 can be used to limit resolved addresses to IPv4 only and bypass these issues.
1161 When uploading, a redirect can cause a local (or remote) file to be
1162 overwritten.  Apps must not allow any unsanitized URL to be passed in for
1168 information to be sent to an unknown second server.  Apps can mitigate against
1172 Use of the CURLAUTH_ANY option to \fICURLOPT_HTTPAUTH(3)\fP could result in
1173 user name and password being sent in clear text to an HTTP server.  Instead,
1177 Use of the CURLUSESSL_TRY option to \fICURLOPT_USE_SSL(3)\fP could result in
1178 user name and password being sent in clear text to an FTP server.  Instead,
1179 use CURLUSESSL_CONTROL to ensure that an encrypted connection is used or else
1184 performs some malicious action to a site whose authentication is already
1193 Apps must not allow unsanitized SCP: URLs to be passed in for downloads.
1196 A malicious server could cause libcurl to effectively hang by sending a
1200 be used to mitigate against this.
1202 A malicious server could cause libcurl to effectively hang by starting to send
1205 set the TCP SO_KEEPALIVE option to mitigate against this.  Setting one of the
1208 A malicious server could cause libcurl to download an infinite amount of data,
1209 potentially causing all of memory or disk to be filled. Setting the
1210 \fICURLOPT_MAXFILESIZE_LARGE(3)\fP option is not sufficient to guard against
1221 \fICURLOPT_POSTFIELDS(3)\fP and others that are used to generate structured
1223 user to create additional headers or fields that could cause malicious
1229 using the Content-disposition: header to generate a file name.  An application
1230 could also use CURLINFO_EFFECTIVE_URL to generate a file name from a
1231 server-supplied redirect URL. Special care must be taken to sanitize such
1232 names to avoid the possibility of a malicious server supplying one like
1237 option to disable certificate validation. There are numerous attacks that are
1238 enabled by apps that fail to properly validate server TLS/SSL certificates,
1239 thus enabling a malicious server to spoof a legitimate one. HTTPS without
1246 risks. Host names, user names, paths, operating system specifics, etc. (not to
1247 mention passwords of course) may in fact be used by intruders to gain
1250 Be sure to limit access to application logs if they could hold private or
1264 The multi interface, on the other hand, allows your program to transfer
1265 multiple files in both directions at the same time, without forcing you to use
1268 interface allows a single-threaded application to perform the same kinds of
1280 of how to use the easy interface. The multi interface is simply a way to make
1286 multi handle with \fIcurl_multi_init(3)\fP and add all those easy handles to
1294 done now and then return back control to your program. It is designed to never
1295 block. You need to keep calling the function until all transfers are
1299 file descriptors or sockets to know when to call libcurl again. This also
1300 makes it easy for you to wait and respond to actions on your own application's
1301 sockets/handles. You figure out what to select() for by using
1306 action and you then call \fIcurl_multi_perform(3)\fP to allow libcurl to do
1307 what it wants to do. Take note that libcurl does also feature some time-out
1308 code so we advise you to never use very long timeouts on select() before you
1310 provided to help you get a suitable timeout period.
1316 If you want to stop the transfer of one of the easy handles in the stack, you
1317 can use \fIcurl_multi_remove_handle(3)\fP to remove individual easy
1324 \fIcurl_multi_info_read(3)\fP can be used to get information about completed
1325 transfers. It then returns the CURLcode for each easy transfer, to allow you
1336 When you add easy handles to a multi handle, these easy handles will
1341 subsequent name resolving faster, and the connection pool that is kept to
1347 example cookies so the only way to share that is with the share interface.
1351 libcurl 7.10.3 and later have the ability to switch over to chunked
1363 This behavior was different in versions before 7.17.0, where strings had to