Special

Introducing the “Welcome to Xojo” Bundle!

New to Xojo and looking for guidance? We've put together a terrific bundle to welcome you! Xojo Bundle

This bundle includes six back issues of the magazine -- all of year 14 in printed book and digital formats -- plus a one-year subscription so you'll be learning all about Xojo for the next year. It's the perfect way to get started programming with Xojo. And you save as much as $35 over the non-bundle price!

This offer is only available for a limited time as supplies are limited, so hurry today and order this special bundle before the offer goes away!

Article Preview


Buy Now

PDF:

Feature

Implementing WinHTTP in REALBasic

Navigating through proxy-servers using Windows native API

Issue: 7.2 (January/February 2009)
Author: Mattias Sandstrom
Article Description: No description available.
Article Length (in bytes): 33,025
Starting Page Number: 24
RBD Number: 7210
Resource File(s): None
Related Web Link(s):

tangix.msa-at-gmail.com

Known Limitations: None

Excerpt of article text...

During my years programming in REALbasic I have always been able to achieve my goals using the native framework or third-party plugins. Commercial plugins from Monkeybread Software or Einhugur have provided functionality to many of my projects and delivered functionality not available in the REALBasic framework. Investing in third-party plugins has been an economical way to focus your most valuable asset -- time -- on those things that makes your product a success and to leverage the knowledge of others. Background My company is developing a companion product to a Windows-only application and the rationale of using REALbasic for Windows-only software is to avoid the problems of installation and relying on other runtimes and frameworks. (Unfortunately, the current version of REALbasic no longer creates single-EXE applications, but discussing the single-EXE approach is opening a big can of worms and I will close this debate for this article by saying that during the last two years our users have been able to run our software on more than 25,000 computers without any problems.) The software itself is used to get certified and shows hands-on questions requiring the use of the main product to answer. Our application is a simple fire-and-forget app only used when taking the certification questions and you should ideally only need to use it once (see Figure 1 for a screenshot of the app). About 18 months ago some of the users started to ask why the software did not support certain kind of proxies when connecting over the Internet. (See the "What is a proxy?" sidebar for more on proxies.) Our software was then using the standard HTTPSecureSocket from the REALbasic framework to communicate with a central server located on the Internet and the communication needs to pass the customers' proxy-solutions. When the HTTPSecureSocket finally supported proxies using the HTTPProxyAddress and HTTPProxyPort properties we were able to include proxy support in the software, supporting proxies that are either transparent to the user or using basic user-authentication. Some problems with the AuthenticationRequired event not firing properly required some workarounds, but the implementation was rather clean despite this. Proxies to support However, some customers were protected by proxies requiring stronger user-authentication than basic authentication or were using the Web Proxy Autodiscovery Protocol (WPAD) to configure their proxy settings (see Code Listing 1 for a sample WPAD configuration file, and Figure 2 for sample proxy settings). WPAD is a great way for system administrators to provide transparency for the user as the proxy configuration is done automatically, but supporting WPAD requires downloading and running a JavaScript for each URL request to get the proxy configuration. Such proxies typically rely on Integrated Windows Authentication where the user's Windows-login credentials are used for authentication; this ensures maximum transparency to the user. The location of the WPAD URL could be either entered into the Registry, supplied using DHCP, or using a DNS A-record in the current DNS-zone. To protect credentials, transmission between the client and the proxy is encrypted and using the NT LAN Manager (NTLM) or Kerberos protocol. As many of the commercially available proxies are using Integrated Windows Authentication, typically using Active Directory to hold the credentials, the only feasible long-term solution was to provide support for NTLM and Kerberos authentication to proxies. Spending numerous hours searching the REALbasic user forums, blogs, and mail-groups without any luck, I broadened the search trying to find anything that could provide a solution or implementation details. Ideally I would like to have stumbled across an implementation in REALbasic, but the chances of this quickly diminished. Researching further, I found out that the NTLM protocol is proprietary to Microsoft but much work has been put into reverse-engineer the protocol to provide support for several open-source products. My first approach to implement NTLM in REALbasic with the help of the clues from the open-source projects (written in C and Python) seemed promising. First attempt: native REALBasic implementation The basic idea is to encrypt the user credentials using the NTLM protocol and setting the information in the HTTP headers sent between the client and the proxy. Sounds easy enough.... The first major obstacle is that NTLM uses MD4 for message hashing and DES encryption of the information. As a user of the e-CryptIt plugin from Einhugur I initially did not see any problems with this. The hard way I found out that MD4 is not the same as MD5 and that the old 64-bit implementation of DES is not used by any sane programmer and thus not available in RB or in the e-CryptIt plugin. The specifications of these algorithms are available on the Internet and working implementations of both were created in two days. With this I was able to craft NTLM compatible HTTP headers -- wow! Trying to get the implementation to work it dawned on me that NTLM is connection-oriented and uses a three-way handshake between the client and the proxy! As the native HTTPSecureSocket implementation is using HTTP/1.0, the connection is closed every time a request has been completed, not allowing me perform the handshake properly and also forcing me to redo the authentication for each request. Getting the native HTTPSecureSocket not to close the connection (and work in a more HTTP/1.1 way) was not possible. A solution I really considered was to program my own incarnation of the HTTP/1.1 protocol using the native TCPSocket but the amount of work to properly implement HTTP/1.1 using TCPSocket was daunting and I started look for another solution. Second attempt: ActiveX Control Researching the proxies that our customers were having problems with, I downloaded evaluation copies of Microsoft IAS 2006 and WebMarshal, both Windows-based proxy applications and started to see how other software handles them. I also installed a Linux-system running the open-source squid proxy. Most of the software running on my machine negotiated the proxies properly, authenticating and parsing the WPAD information properly, so why couldn't I do that? Digging through the Microsoft Developer Network (MSDN) I found information about the WinHTTP 5.1 library of functions promising to solve all problems with both WPAD and NTLM! I was thrilled when I saw the ActiveX version of WinHTTP and that REALbasic supported this using "Add ActiveX component..."! Trying this out I easily got a working implementation using the WinHTTP ActiveX component and was able to verify compatibility with the various proxies. However, two problems with the ActiveX implementation were found: The ActiveX implementation does not support asynchronous mode of operation and thus no feedback to the user is possible during a lengthy download. (Even though there is an EventTriggered handler on the REALbasic object, this does not work and does not provide any feedback during operation.) The reason for this problem is how the ActiveX object is implemented. Another major disadvantage of the ActiveX interface is that WPAD is not supported. After trying the basic functionality of the ActiveX, I decided not to implement WinHTTP using the ActiveX approach. To achieve the results I wanted, the only option seemed to be using the WinHTTP API directly. Third attempt: native WinHTTP API Going down the path to create my own implementation, I started browsing the MSDN documentations on WinHTTP. The documentation provides a pretty nice combination of articles ranging from a high-level 10,000 feet overview of the functions of WinHTTP down to the nitty-gritty details of all individual API calls. This combination allows a developer to quickly get the overview and then code the individual pieces as they are needed. During the project this was very valuable as it made testing of individual components easier. A couple of examples in C++ also provided a jump-start to the coding, especially since most of the code consists of memory-management functions (allocating and de-allocating) and the API-calls themselves. Converting these C++ snippets to REALbasic was easy following the standard rules of converting API calls to REALbasic (documented by Aaron Ballman's articles for example). While getting acquainted with WinHTTP the major hurdles of asynchronous operation were identified: handling callbacks from the API and exchanging data with the API during asynchronous operation. MemoryBlocks and variables containing structures must not get out of scope while waiting for the asynchronous operation to complete -- the same for the structures containing data. Several API-calls also return structures containing pointers to where the information is stored, memory that needs to be managed properly and is not subject to the garbage collection and memory freeing of REALbasic. Once started, the programming was pretty straight-forward. Using one of the C++ examples as a base, basic functionality was quickly achieved implemented as synchronous calls from within a Sub. This was an excellent opportunity to see how the API functions behave and what needs to be configured for proper operation. WinHTTP holds an impressive amount of Structures and Constants which takes quite some time to port to REALbasic, especially as the user interface of the REALbasic IDE is inefficient when it comes to the number of clicks required to define each Constant (see Figure 3). I cheated here by creating a small hack converting the #define statements of the Constants from the WinHTTP header-files into XML, exported the RB project as XML, pasted the information into the XML-file, and then re-opened the project in the REALbasic IDE. The Structures needed were configured using the IDE, as parsing the typedef struct statements from the header-files was far more complicated than parsing the #define statements. Using WinHTTP Using the WinHTTP API to send and receive data involves about a dozen API-calls, all handling certain aspects of the communication such as querying proxy settings, setting up the connection, authenticating, preparing the request, setting the headers, sending the request, and finally receiving the response. Requests to the same host can be handled in the same connection as HTTP/1.1 is used by WinHTTP and is recommended for best performance (the HTTP/1.1 specifications reduce the need to tear-down and setup the connection for each request). To be able to keep track of the calls to WinHTTP, three different handles are used; one for the entire session tracking the library (hSession), one for the connection between the client and the server (hConnect) and one for each request sent over the connection (hRequest). To give an idea of how complex WinHTTP is to use, a complete transaction requesting a web-page will be described here including the API calls and some details about the REALbasic implementation. All API calls are declared "Soft" to make sure that we run properly on non-WinHTTP enabled operating systems (pre-Windows 2000). However, Windows 2000 contained version 5.0 of WinHTTP (although there is an update to WinHTTP 5.1 available for Windows 2000 users) with some notable differences making it necessary to create a separate implementation for Windows 2000. Therefore we decided to stop supporting Windows 2000 due to this incompatibility between version 5.0 and 5.1. The flow of calls is the same regardless if WinHTTP is used synchronous or asynchronous. In synchronous mode, most of the API calls will return after they are done and there is no need for callback handlers. Starting the implementation, I aimed for asynchronous mode of implementation, but as we will later learn I was not able to get this to work properly. All declares, structures, and constants were obtained from the Windows Platform Software Development Kit (SDK) installed before starting this project. Depending on the version of the SDK (I used the SDK for Windows 2003 Server) some details were missing that I needed to Google for -- for example, some error constants that were not defined in my installation. The whole SDK is not needed, but the file winhttp.h from the SDK is a crucial piece of information to have. The SDK is available from the Microsoft Download Center in many flavors and installation packages. The complete documentation to each of the API calls is available on MSDN, simply Google winhttp functionname and the first hit is very likely to be to MSDN. Be sure to check out the other hits as well as they may explain common pit-falls and problems you are likely to see. WinHttpOpen This is the first call to WinHTTP and initializes the library for operation. A new session context is created and stored as hSession. The API documentation states that it is possible to have multiple invocations of WinHTTP but this leads to unexpected crashes and I was forced to revert this and only have one WinHTTP library open. Soft Declare Function WinHttpOpen Lib "winhttp.dll" (pwszUserAgent as WString, dwAccessType as Integer, pwszProxyName as Integer, pwszProxyBypass as Integer, dwFlags as Integer) as Integer WinHttpOpen returns an integer handle to the opened library and will be used in all operations to WinHTTP. The call contains several parameters configuring how WinHTTP should operate; most notably we need to specify a UserAgent string in pwszUserAgent and set dwAccessType to indicate how a network proxy should be handled. Soft Declare Function WinHttpSetOption Lib "winhttp.dll" (hInternet as integer, dwOption as integer, Byref lpBuffer as integer, dwBufferLength as integer) as boolean The WinHttpSetOption can be used to modify operation of the WinHTTP library and is used immediately after the WinHttpOpen call to disable trace functionality (unless this is a DebugBuild). To secure the communication, see more about tracing WinHTTP later in this article. ... #if not DebugBuild dwFlags = 0 if not WinHttpSetOption(hSession, WINHTTP_OPTION_ENABLETRACING, _ dwFlags, 4) then dwFlags = GetLastError end if #endif ... WinHttpConnect When WinHTTP has been opened, the next step is to open the connection to the remote host. This is done with a call to the WinHttpConnect function. Soft Declare Function WinHttpConnect Lib "winhttp.dll" (hSession as integer, pswzServerName as WString, nServerPort as integer, dwReserved as integer) as integer Here it may seem easy to simply set the URL directly (i.e. https://www.company.com/resource.html) but this is definitely not the case! As this is the connection handle, only the hostname of the URL should be used and this leads to an intricate problem -- how to parse a URL and identify the various parts of an URL (hostname, port, protocol, username, password and resource identifier)? Consider the URL ftp://username:password-at-host.company.com:2121/file.txt. Parsing this URL will require you to read up on RFC1738 rather extensively to successfully identify all the parts. Fortunately there is a function in WinHTTP available to "crack" the URL into the various parts -- WinHttpCrackUrl: Soft Declare Function WinHttpCrackUrl Lib "winhttp.dll" (pwszUrl as Ptr, dwUrlLength as integer, dwFlags as integer, ByRef lpUrlComponents as URL_COMPONENT) as boolean Unfortunately using WinHttpCrackUrl is not that easy when you start looking at the implementation details. Feeding the WinHttpCrackUrl function with the URL (as a pointer to a MemoryBlock encoding the URL in UTF-16) will return a URL_COMPONENT structure and now the fun begins! The URL_COMPONENT contains member properties HostName, Password, Port, Scheme, UrlPath, and UserName that are all needed in the following steps. Finally, passing hSession, the HostName (as UTF-16), and Port of the remote server to WinHttpConnect will prepare WinHTTP for the connection and return a connection handle (hConnect). WinHttpOpenRequest We are now ready to prepare the actual request (the Verb GET or POST and the standard specifies). Using the WinHttpOpenRequest function provides a way to create this request. Soft Declare Function WinHttpOpenRequest Lib "winhttp.dll" (hConnect as integer, pwszVerb as WString, pwszObjectName as WString, pwszVersion as WString, pwszReferrer as WString, ppwszAcceptTypes as WString, dwFlags as integer) as integer Specifying hConnect (that is a link to hSession via the previous call to WinHttpConnect), the Verb and the resource to get will return a handle to the actual request, hRequest. Worth noting here is the dwFlags needs to be set to specify if SSL should be used for the request. Up until now we have not specified any proxy specifications, but do not think that WinHTTP will work the magic for you... No way! Using the handle to the actual request (hRequest) it is now time to start setting options on that request. The WinHTTP specifications say that you should check proxy settings for each request and not rely on previous results as the network administrator may have load-balancing firewalls installed. Setting the WPAD proxy requires a call to the Registry to get the WPAD proxy URL and feed that into the WinHttpGetProxyForUrl function. Soft Declare Function WinHttpGetProxyForUrl Lib "winhttp.dll" (hSession as integer, lpcwszUrl as Ptr, ByRef pAutoProxyOptions as WINHTTP_AUTOPROXY_OPTIONS, ByRef pProxyInfo as WINHTTP_PROXY_INFO ) as boolean Two rather complex structures are needed now; the WINHTTP_AUTOPROXY_OPTIONS contains information about the proxy configured and the magic flag fAutoLogonIfChallenged that will make it possible to negotiate WPAD proxies that have their WPAD configuration protected by a login (a nice Catch 22). AutoDetectFlags in WINHTTP_AUTOPROXY_OPTIONS also specifies how the WPAD configuration should be detected (entry in the Registry, DHCP, or DNS A-Record). The other structure WINHTTP_PROXY_INFO is the return value from the function containing information on how the request should negotiate any proxies to get to the server. Using the returned WINHTTP_PROXY_INFO, this is set as options to the request using WinHttpSetOption. WinHttpAddRequestHeaders As we are implementing our own web-client, we need to obey the standards of HTTP/1.1 and set certain headers in the request. The function WinHttpAddRequestHeaders is used when setting headers, specifying the hRequest and then the header information (for example "Content-Length" or "Content-Type") Soft Declare Function WinHttpAddRequestHeaders lib "winhttp.dll" (hRequest as integer, pwszHeaders as WString, dwHeadersLength as integer, dwModifiers as integer) as boolean When using HTTP/1.1 we need to set at least the Content-Length header to comply with the standards. WinHttpSendRequest To recap so far; we have opened WinHTTP, cracked the URL, created a connection handle, and a new request. Furthermore, we have configured the proxy settings for the request and added the proper headers. Finally we have arrived to the point where the request is actually leaving the host going out on the network! The function WinHttpSendRequest will take the handle to the requests, allow us to set additional headers and the length of the optional data of the request. Soft Declare Function WinHttpSendRequest lib "winhttp.dll" (hRequest as integer, pwszHeaders as WString, dwHeadersLength as integer, lpOptional as Ptr, dwOptionalLength as integer, dwTotalLength as integer, dwContext as Ptr) as boolean Optional data? Yes, a POST request contains the data from the form encoded using the variable=value method in the body of the request (i.e. appended after the headers). A POST request also requires the Content-Type header to be set to "application/x-www-form-urlencoded" which must be done using the WinHttpAddRequestHeaders function. WinHttpReceiveResponse Using the function WinHttpReceiveResponse, we indicate that we are ready to start receiving data from the server. Soft Declare Function WinHttpReceiveResponse lib "winhttp.dll" (hRequest as integer, lpReserved as Ptr) as boolean Communicating with a server can lead to two interesting problems; either the proxy requires authentication (indicated with a 407 HTTP Response) or the SSL certificate can not be validated (i.e. a self-signed certificate). Problematic SSL certificates is very common with the IAS proxy as that proxy decrypts all SSL-encrypted traffic by terminating them on the inside and then creates a new connection with the external server. As the clients connecting to the IAS proxy generally trust the root-certificate of the ActiveDirectory domain, a casual user will not notice anything except if they check the certificate for the remote server and the name of the Common Name). The SSL Certificate problem is reported immediately by the call to WinHttpReceiveResponse with the ERROR_WINHTTP_SECURE_FAILURE value. If this happens, we need to tear down the connection, restart with WinHttpConnect, set the options on the request with WinHttpSetOption, and send the request again. Here it was quite tricky to find the combination of ignore flags to use and the combination that worked for my self-signed certificate was (where the flag SECURITY_FLAG_IGNORE_CERT_WRONG_USAGE was the trickiest to find): SECURITY_FLAG_IGNORE_CERT_CN_INVALID SECURITY_FLAG_IGNORE_CERT_DATE_INVALID SECURITY_FLAG_IGNORE_CERT_WRONG_USAGE SECURITY_FLAG_IGNORE_UNKNOWN_CA WinHttpQueryHeaders, WinHttpQueryAuthSchemes and WinHttpSetCredentials Handling proxy authentication requires being able to detect the 407 header response, which should be done using WinHttpQueryHeaders that is available after WinHttpSendRequest succeeded: Soft Declare Function WinHttpQueryHeaders Lib "winhttp.dll" ( hRequest as integer, dwInfoLevel as integer, pwszName as WString, lpBuffer as Ptr, byref lpdwBufferLength as integer, lpdwIndex as integer) as boolean Getting a 407 header now requires a way to detect what kind of authentication methods available to use on the proxy server. Using the WinHttpQueryAuthSchemes function will return Kerberos, NTLM, Basic, Passport or Digest authentication Soft Declare Function WinHttpQueryAuthSchemes Lib "winhttp.dll" (hRequest as integer,byref lpdwSupportedSchemes as UInt32, ByRef lpdwFirstScheme as UInt32, ByRef pdwAuthTarget as UInt32) as boolean The trick is now to select the most secure scheme supported and set the credentials using the WinHttpSetCredentials function Soft Declare Function WinHttpSetCredentials Lib "winhttp.dll" (hRequest as integer, AuthTargets as integer, AuthScheme as integer, pwszUserName as WString, pwszPassword as WString, pAuthParams as Ptr) as boolean If credentials are needed for the proxy, you are responsible to provide the user a way to enter this. Remember, though, that the authentication scheme deployed may involve the use of smartcards, thumbprints, or other means of authentication. Fortunately there is an API available on Windows XP or later, CredUIPromptForCredentialsW that will handle all this. Soft Declare Function CredUIPromptForCredentialsW Lib "credui.dll" (ByRef pUiInfo as CREDUI_INFO, pszTargetName as Ptr, Reserved as Ptr, dwAuthError as integer, pszUserName as Ptr, ulUserNameMaxChars as integer, pszPassword as Ptr, ulPasswordMaxChars as integer, ByRef pfSave as integer, dwFlags as integer) as integer Once again we are facing a rather complex API call with lots of MemoryBlocks to pass and get result into. WinHttpQueryDataAvailable and WinHttpReadData If everything is OK, we are now receiving data that is buffered by WinHTTP in blocks of 8192 bytes ready to move to another buffer and then poll WinHTTP again for another block of information. Soft Declare Function WinHttpQueryDataAvailable Lib "winhttp.dll" (hRequest as integer, lpdwNumberOfBytesAvailable as Ptr) as boolean Data is transferred from the WinHTTP buffer to a MemoryBlock using WinHttpReadData. Soft Declare Function WinHttpReadData lib "winhttp.dll" (hRequest as integer, lpBuffer as Ptr, dwNumberOfBytesToRead as integer, ByRef lpdwNumberOfBytesRead as integer) as boolean The loop is pretty straight forward and implemented as a separate Thread until all data has been received. do dwSize = SyncQueryData if dwSize > 0 then lpszOutBuffer = new MemoryBlock(dwSize+2) if WinHttpReadData(hRequest, lpszOutBuffer, dwSize, dwDownloaded) then me.dataReceived = me.dataReceived + lpszOutBuffer.StringValue(0,dwSize) dwBufferSize = dwBufferSize + dwSize me.bytesReceived = me.bytesReceived + dwSize end if end if loop until dwSize = 0 From the headers received with the WinHttpQueryHeaders function, we know how much data to expect (assuming that the server sending the data specifies this) and we keep an internal tally using bytesReceived of how much data has been received. WinHttpCloseHandle After a request has been completed, the request handle (hRequest) must be closed using WinHttpCloseHandle Soft Declare Function WinHttpCloseHandle Lib "winhttp.dll" ( hInternet as integer ) as boolean Implementing as a Class If additional requests are destined for the same server, a new request is started with a new call to WinHttpOpenRequest and can be processed directly. If not, the connection handle (hConnect) and the session (hSession) must be properly closed to clean up after the request. This suits the implementation as a separate class rather well: the Constructor opens the WinHTTP library, gets the hSession handle, and configures proxy settings by making calls to different APIs. The Destructor will clear all resources allocated and close each handle as needed by the library. Implementing Get and Post methods means handling the rest of the communication using the hSession handle involved connecting to the remote server, checking for invalid SSL certificates, authenticating to the proxy server (if needed), and finally calling WinHttpReceiveResponse to start receiving the data When the synchronous operation was working, the conversion to asynchronous operation started. This also required a conversion from the Sub-style of coding to a stand-alone Class, this to simplify the process of replacing the HTTPSecureSocket-derived class of the original application. Some of the common functions of the old class were mimicked in the WinHTTP-based class and converted into the WinHTTP equivalent. Each separate API call was converted into Functions or Subs in the class and some common variables were defined. General functions, such as Get and Post were implemented quickly and was rapidly made to work. Implementation The clean implementation I would have liked to achieve was not possible due to two shortcomings in REALbasic: Structures are not possible to define in a Class, but must be defined in a Module (this is a feature introduced in REALbasic 2008r3). Furthermore, Class Methods may not be used as Callbacks -- Shared Methods must be used instead. Using a Shared Method was a first for me and I quickly found it a bit cumbersome to use as the me keyword can not be used from a Shared Method. Trying to find a way round this, an ugly kludge had to be made where the Constructor of the Class sets a variable to use instead of me. As we will only have one instance of the Class, this will work fine despite this. Asynchronous problems! Around a week into this part of the project, the code from the test-bed was migrated into the target software and I now started to notice intermittent failures during communication. I ran into problems with the locking mechanism (IllegalLockingException) used when protecting the callback and also stack problems (StackOverFlowException) -- sigh! Clearly the initial approach from the test-bed did not work very well in a more complex environment and needed to be reconsidered. Looking further into the code, which is hard as RB does not allow break-points in callback routines to work properly, the reason for the failure was due to the callback routine. A major problem was identified where the callback happens so fast that the call to the API has just been finished and thus the previous callback has not been finished, with the stack rapidly exhausted causing the StackOverFlowException to happen. With the broken locking implementation (calling from different threads), I needed a new approach to implement this. Putting my thinking hat on, the practical solution I came up to was to use WinHTTP in synchronous mode, but run the receiving of the data as a separate Thread polling the WinHTTP API for new data. Therefore, sub-classing the Thread class and adding the receiving loop in the Run event of the Thread solved the problem of giving the user feedback while downloading the information. As data is received using the WinHttpReadData API call, it is added to a buffer and the number of bytes received is incremented allowing feedback to the user during the download. Conclusions This is how the communication is now implemented and has been working rather well during the last year. The software is able to negotiate all of the proxies that have been thrown upon it in some really interesting proxy configurations. We see some occasional problems with heavily loaded proxies that do not reply properly and shut down the TCP-connection, causing a bunch of errors in WinHTTP that we need to try to troubleshoot using the log- and trace-files. Dealing directly with WinHTTP has generated some number of crashes related to the proxy configuration of the system. For example, entering a space before the WPAD URL. Sending data to the API requires well-formatted data according to the specification or else... With the introduction of WinHTTP into the software we have gone completely away from being cross-platform. For the moment this is not a big issue but if future demands on the software include cross-platform, we will definitely be in problems. Reflections Trying to summarize this rather daunting and challenging task is hard but I'd like to point out some resources that have helped me during the project. API Documentation and RB porting Make sure you freshen up your API-to-RB porting knowledge to the point that you are able to read the C++ examples and instinctively know that a variable lpcwszUrl is a pointer to a null-terminated UTF-16 encoded string -- and know that the RB equivalent is a MemoryBlock having WStrings added to it. Aaron Ballman's guide on porting WinAPI calls to RB is mandatory reading together with the source of his WindowsFunctionalitySuite. There is also lots of VisualBasic information on the Internet with examples on how to access API functions that may help you translate the "Hungarian notation" from the API documentation (like lpcwszUrl) to RB code. A fantastic source of information was from sites discussing using Windows API from the Delphi programming language where the different variable types were discussed in much detail and how they translate to Delphi variables and from there I could define their proper RB type. Use MSDN and the C++ examples The documentation on MSDN is very extensive but once you get around it, you will find it a good source of information. The C++ examples are really good, whereas the VisualBasic examples tend to be much more simple in functionality and options used. WinHTTP Tracing Built into WinHTTP is a great tool for troubleshooting, namely the WinHTTP tracing facility. This is a powerful tool generating a text file per process with all the details on how that process is using WinHTTP. When I say "all details" I really mean all details: you will get access to the headers and decoded HTTP streams (including decrypted HTTPS streams), thus the importance to disable the trace facility in you own application using WinHttpSetOption. Proxy Testing Tools Make sure you have VMware Workstation running where you can install the different proxy products you are targeting. Downloading evaluation copies and sometime simply requesting a license for development work to support a vendor's proxy can work (yes, some of the vendors are happy to supply you with a license). You need separate physical machines to install them on unless you have a virtualization software. A valuable tool was the Charles Web Debugging Proxy (see Figure 4) allowing me to peek into the data stream and simulating proxies. WireShark was another valuable tool for this, but Charles is better targeted for this kind of development and testing. Web Resources: http://delphi.about.com/od/windowsshellapi/l/aa101303a.htm http://msdn.microsoft.com http://msdn.microsoft.com/en-us/library/aa384240.aspx http://www.aaronballman.com/programming/ http://www.charlesproxy.com/ http://www.ietf.org/rfc/rfc1738.txt http://www.wireshark.org/

...End of Excerpt. Please purchase the magazine to read the full article.

Article copyrighted by REALbasic Developer magazine. All rights reserved.


 


|

 


Weblog Commenting and Trackback by HaloScan.com