This is a continuation of multi-part series on how to create a cross-platform desktop video downloading application. In part I we prepared our coding environment with vim, various vim plugins, mingw and gdb. In this part we will start creating a C library for downloading just the YouTube videos with modularity in mind, so that we could create a nice desktop application capable of downloading videos from other video sharing sites like break, metacafe, dailymotion and others. Complementary to this library we will be also be creating a command line application which will use this C library. The command line application will serve us as a test tool for the library, so we know the library works ok. Once we have these two components, we will build a GUI around the library and create the desktop app.
Writing the Library
I have chosen a name libvd for the library which stands for video download library.
The library will be written in C programming language because:
- it is the number one cross platform language,
- I haven't programming in C for a while, so it will be a great refresher,
- the GUI framework which I will be using is written in C++ which will have no problems with using C code
- the application will be small and will require no unnecessary library/toolkit dependencies
Downloading a video means working with HTTP protocol. Let's use "good coders code, great reuse" approach and seek for a C library which would ease the usage of HTTP protocol.
Having worked on a few projects in C programming language involving HTTP protocol, I know an excellent library. It's called libcurl. I even implemented cookie interface for this library which was missing two years ago.
What is libcurl? Quoting the author of libcurl: "libcurl is a free and easy-to-use client-side URL transfer library, supporting FTP, FTPS, HTTP, HTTPS, SCP, SFTP, TFTP, TELNET, DICT, FILE and LDAP. libcurl supports SSL certificates, HTTP POST, HTTP PUT, FTP uploading, HTTP form based upload, proxies, cookies, user+password authentication (Basic, Digest, NTLM, Negotiate, Kerberos4), file transfer resume, http proxy tunneling and more! libcurl is highly portable, it builds and works identically on numerous platforms, including Solaris, NetBSD, FreeBSD, OpenBSD, Darwin, HPUX, IRIX, AIX, Tru64, Linux, UnixWare, HURD, Windows, Amiga, OS/2, BeOs, Mac OS X, Ultrix, QNX, OpenVMS, RISC OS, Novell NetWare, DOS and more..."
Woah! Much more than we need! And notice how cross platform it is! Exactly what we are looking for!
Let's download the libcurl library. Since I am developing the program on a fresh Windows XP virtual machine, I chose to download curl-7.16.4.zip version of the library. It really does not matter what operating system I am developing this program on, I could have as well chosen Linux, then I would have downloaded the library ending with extension .gz or .bz2.
Compiling libcurl
Once we have downloaded libcurl, let's unpack it and look for instructions on how to compile it with MinGW on a Windows machine. I chose c:\vd as the root directory for the project on the windows machine because I am afraid of those whitespace characters windows has in path. I unpacked the libcurl library to c:\vd\curl.
At this point we have no idea how to build the libcurl on Windows, right? So let's look in the docs subdirectory (C:\vd\curl\docs) for a clue. Usually the instructions on how to build a library on a particular platform can be found in files like README.platform or INSTALL.platform or just INSTALL. Indeed, libcurl has a file README.win32. Let's read it.
Oops, it says to read README first, we look in the same docs subdirectory but there is no README! We look in the parent directory, whew, yeah there is a README. But it is just 1KB in size and says nothing...
The only reasonable file left is INSTALL, let's look at it. Oh yeah, it's 29K in size! Must be it!
Indeed, scrolling a few screens down by pressing vim's Ctrl_D, we find instructions on how to build libcurl on win32 with MinGW compiler!
Libcurl also provides support for compressed HTTP content with zlib, and secure connections over HTTPS with OpenSSL. The INSTALL file mentions what to do to get support for these thingies. We do not need them, so we ignore them.
Don't rush yet compiling the library! We also want our application to be small, not just portable. Libcurl has support for so many protocols but our project need just the HTTP protocol!
Looking a little further in the INSTALL file, we find the following instruction:
Disabling Specific Protocols in Win32 builds -------------------------------------------- The configure utility, unfortunately, is not available for the Windows environment, therefore, you cannot use the various disable-protocol options of the configure utility on this platform. However, you can use the following defines to disable specific protocols: <strong>HTTP_ONLY</strong> disables all protocols except HTTP [...] If you want to set any of these defines you have the following possibilities: - Modify lib/setup.h [...]
We really want just the HTTP protocol, so let's follow this instruction and modify setup.h file in lib directory (file C:\vd\curl\lib\setup.h) and add the following line right after the comments:
#define HTTP_ONLY
The file should now look like this:
Okay, now we are ready to compile libcurl!
The INSTALL file says that to compile the libcurl library we first need to run 'mingw32.bat' and then 'make mingw32'.
MingW32 ------- Run the '<strong>mingw32.bat</strong>' file to get the proper environment variables set, then run '<strong>make mingw32</strong>' in the root dir. Use 'make mingw32-ssl' to build curl SSL enabled.
mingw32.bat? I don't have such file in my mingw32 distribution! Now what? Let's be real developers and compile the library ourselves then. Let's make sure the MinGW's bin directory (in my case C:\MinGW\bin) is in our PATH environment variable and let's run the second command - 'make mingw32'.
Oops:
C:\vd\curl>make mingw32 'make' is not recognized as an internal or external command, operable program or batch file.
There is no program named 'make' in MinGW's distribution, there is only mingw32-make.exe, let's run it:
C:\vd\curl>c:\MinGW\bin\mingw32-make.exe mingw32 c:/MinGW/bin/mingw32-make -C lib -f Makefile.m32 ZLIB=1 mingw32-make[1]: Entering directory `C:/vd/curl/lib' gcc -I. -I../include -I"../../zlib-1.2.3" -g -O2 -DBUILDING_LIBCURL -DHAVE_LONGLONG -DHAVE_LIBZ -DHAVE_ZLIB_H -c file.c gcc -I. -I../include -I"../../zlib-1.2.3" -g -O2 -DBUILDING_LIBCURL -DHAVE_LONGLONG -DHAVE_LIBZ -DHAVE_ZLIB_H -c timeval.c gcc -I. -I../include -I"../../zlib-1.2.3" -g -O2 -DBUILDING_LIBCURL -DHAVE_LONGLONG -DHAVE_LIBZ -DHAVE_ZLIB_H -c base64.c In file included from base64.c:43: urldata.h:92:59: zlib.h: No such file or directory In file included from base64.c:43:
Huh? I never wanted my libcurl to be built with zlib support. Let's get rid of it by editing the Makefile (C:\vd\curl\Makefile) directly. Search for mingw32 target (string "mingw32:") and you'll find:
mingw32:
$(MAKE) -C lib -f Makefile.m32 ZLIB=1
$(MAKE) -C src -f Makefile.m32 ZLIB=1
We really don't want to build the curl executable, so let's get rid of the last line and also get rid of "ZLIB=1" variable definition.
The lines should look now like this:
mingw32:
$(MAKE) -C lib -f Makefile.m32
Now let's run the mingw32-make again:
C:\vd\curl>c:\MinGW\bin\mingw32-make.exe mingw32 c:/MinGW/bin/mingw32-make -C lib -f Makefile.m32 mingw32-make[1]: Entering directory `C:/vd/curl/lib' gcc -I. -I../include -g -O2 -DBUILDING_LIBCURL -DHAVE_LONGLONG -c base64.c gcc -I. -I../include -g -O2 -DBUILDING_LIBCURL -DHAVE_LONGLONG -c hostip.c [...] rm -f libcurl.a process_begin: CreateProcess(NULL, rm -f libcurl.a, ...) failed. make (e=2): The system cannot find the file specified. mingw32-make[1]: *** [libcurl.a] Error 2 mingw32-make[1]: Leaving directory `C:/vd/curl/lib' mingw32-make: *** [mingw32] Error 2
Argh! There is no 'rm' on Windows, at least not with MinGW development tools package. Let's modify the Makefile.m32 file (C:\vd\curl\lib\Makefile.m32) and change 'rm' to Windows 'del' command.
Find this line in Makefile.m32:
RM = rm -f
and change it to
RM = del /F
Perfect! Running the mingw32-make.exe again finishes with no errors! Lookin in the C:\vd\curl\lib directory we find the static library libcurl.a and the dynamic library libcurl.dll. Had we compiled it on Linux, the dynamic library would have been named libcurl.so.
Testing libcurl
I wrote the following test program which outputs the website of google.com to the standard output if libcurl works correctly:
#include <stdio.h>
#include <stdlib.h>
#include "curl/curl.h"
int main(void) {
CURLcode ret;
CURL *curl = curl_easy_init();
if (curl == NULL) {
fprintf(stderr, "Failed creating CURL easy handle!\n");
exit(EXIT_FAILURE);
}
/* let's get google.com because I love google */
ret = curl_easy_setopt(curl, CURLOPT_URL, "http://www.google.com");
if (ret != CURLE_OK) {
fprintf(stderr, "Failed getting http://www.google.com: %s\n",
curl_easy_strerror(ret));
exit(EXIT_FAILURE);
}
ret = curl_easy_perform(curl);
if (ret != 0) {
fprintf(stderr, "Failed getting http://www.google.com: %s\n",
curl_easy_strerror(ret));
exit(EXIT_FAILURE);
}
return 0;
}
I placed this program in C:\vd\tests\curltest directory and named the source file testcurl.c. Our curl library is located in C:\vd\curl\lib and the curl header files in C:\vd\curl\include. Assuming out currenly working directory is C:\vd\tests\curltest, the following command line will compile the C source file to an executable:
gcc testcurl.c -o testcurl -DCURL_STATICLIB -I..\..\curl\include -L..\..\curl\lib -lcurl -lwsock32 -lwinmm -Wall
It will create testcurl.exe executable which can be run. Let's try running it:
C:\vd\tests\libcurl>testcurl.exe <HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8"> <title>Google</title> [...]
Yeah! It works! Now we are ready to write the libvd (video download) library which will use the libcurl for communicating over HTTP protocol.
I'll do this in Part III of this series. Until then, have fun compiling libcurl!