Fixing -[NSMutableURLRequest setValue:forHTTPHeaderField:]

UPDATE

This problem is fixed as of Mac OS X 10.8.3+ and iOS 5.1+. I have no idea when it was actually fixed since radar sucks so much.

The problem

From NSMutableURLRequest setValue:forHTTPHeaderField: documentation:

In keeping with the HTTP RFC, HTTP header field names are case-insensitive.

I see three problems with this.

1. Assuming all HTTP implementations are RFC compliant is foolish to say the least.
2. Enforcing case-insensitivity is nonsense.
3. This bit of documentation is accurate, the case of header fields is actually changed.

Trying this

    NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL:[NSURL URLWithString:@"http://www.example.com"]];
    [request setValue:@"MyValue" forHTTPHeaderField:@"MyField"];
    NSLog(@"result: %@", [request allHTTPHeaderFields]);

outputs

    result: {
        Myfield = MyValue;
    }

Notice how the 'F' of MyField was lowercased, that sucks! If the HTTP server you are trying to communicate with is case-sensitive, you are screwed. I filed radar #8029516, which is a duplicate of radar #3131623, which means there is almost no chance to have this fixed anytime soon.

When this happens, you should contact the server administrator asking for a case-insensitive implementation. But in the meantime, if you badly need to preserve the case of your headers, read on.

The investigation

First, let's fire otx to disassemble the Foundation framework in order to have a look at setValue:forHTTPHeaderField: implementation. I strongly suggest you to use otx version from subversion trunk, as it is better at resolving symbols in tail call optimizations (fixed in r553).

Excerpt from otx 0.16b:

   +18  [...]  movl     %eax,0x08(%ebp)
   +21  [...]  leave
   +22  [...]  jmpl     0x002c52bb

Excerpt from otx trunk:

   +18  [...]  movl     %eax,0x08(%ebp)
   +21  [...]  leave
   +22  [...]  jmpl     0x002c52bb       _CFURLRequestSetHTTPHeaderFieldValue

Much better, isn't it?

So let's go through the function calls of setValue:forHTTPHeaderField:. With a basic static analysis of the Foundation and CFNetwork disassemblies, we can draw the following call path:

-[NSMutableURLRequest(NSMutableHTTPURLRequest) setValue:forHTTPHeaderField:]
    CFURLRequestSetHTTPHeaderFieldValue()
        URLRequest::setHTTPHeaderFieldValue()
            HTTPRequest::setHeaderFieldValue()
                CFHTTPMessageSetHeaderFieldValue()
                    HTTPMessage::setHeaderFieldValue()
                        _CFCapitalizeHeader()

_CFCapitalizeHeader looks like a very good candidate for being the bastard changing the case of our headers. A quick search reveals the source code of CFNetwork that was open source long time ago. Although the open source implementation does not exactly match what we see in the disassembly (it is now using toupper instead of ch + 'A' - 'a' for example), we are now absolutely sure that _CFCapitalizeHeader is the function responsible for messing with our headers.

Now, let's check is if there is a path that will not call _CFCapitalizeHeader and if we can somehow influence the condition that would avoid the call to _CFCapitalizeHeader. This is quickly checked, especially if you enabled the Separate logical blocks option of otx (-b option for cli).

HTTPMessage::setHeaderFieldValue(__CFString const*, __CFString const*)
    +0  000515a4  55                      pushl       %ebp
    +1  000515a5  89e5                    movl        %esp,%ebp
    +3  000515a7  83ec28                  subl        $0x28,%esp
    +6  000515aa  8b4508                  movl        0x08(%ebp),%eax
    +9  000515ad  8975f8                  movl        %esi,0xf8(%ebp)
   +12  000515b0  8b7510                  movl        0x10(%ebp),%esi
   +15  000515b3  897dfc                  movl        %edi,0xfc(%ebp)
   +18  000515b6  8945f4                  movl        %eax,0xf4(%ebp)
   +21  000515b9  8b450c                  movl        0x0c(%ebp),%eax
   +24  000515bc  890424                  movl        %eax,(%esp)
   +27  000515bf  e88a26fbff              calll       __CFCapitalizeHeader
   +32  000515c4  c744240cffffffff        movl        $0xffffffff,0x0c(%esp)
   +40  000515cc  89742408                movl        %esi,0x08(%esp)
   +44  000515d0  89c7                    movl        %eax,%edi

We see that there is no path that avoid the call to _CFCapitalizeHeader. So we are left with the last resort solution: patching _CFCapitalizeHeader. With APE Lite, function patching is very easy. You first use APEFindSymbol() to find the address of a non-exported symbol (i.e. __CFCapitalizeHeader), then APEPatchCreate() to replace a function implementation with your own, while still keeping a reference to the original implementation. On iPhone OS, you can use APE Lite+arm (my implementation of the APE Lite API using MobileSubstrate).

The solution

NSMutableURLRequest+CaseSensitive is a category on NSMutableURLRequest that adds these three methods:

   - (void) setAllHTTPHeaderFields:(NSDictionary *)headerFields caseSensitive:(BOOL)caseSensitive;
   - (void) setValue:(NSString *)value forHTTPHeaderField:(NSString *)field caseSensitive:(BOOL)caseSensitive;
   - (void) addValue:(NSString *)value forHTTPHeaderField:(NSString *)field caseSensitive:(BOOL)caseSensitive;

Just pass caseSensitive:YES for preserving the case of your header fields.

Warning: you SHOULD NOT use this in production code. But hey, HTTP implementations SHOULD be case-insensitive ;-)