Parsing HTTP headers in C

Question

I'm new to C and just finished K&R. This is a code that I've written to parse headers of a HTTP request. I want to know if my code:

• Is safe? (after all this is C)
• Does proper exit on failure? Are the ways that exitLog and exitNullPointer functions exit considered good practice?
• Has good implementation structure? Is returning a pointer to a struct and having a helper function to free the allocated memory common? Is having NULL in the header name to indicate end of headers a bad practice?

Here is the code:

#include <stdio.h>
#include <stdlib.h>
#define DEFAULT_HEADER_COUNT 30
#define MAX_HEADER_COUNT 1000
#define DEFAULT_HEADER_FIELD_SIZE 1000
#define MAX_HEADER_FIELD_SIZE 20e6 // 20MB
void exitLog(char *log)
{
    printf("\n%s\n", log);
    exit(1);
}
void exitNullPointer(void *ptr)
{
    if (ptr == NULL)
        exitLog("Malloc/Realloc failed.");
}
struct header
{
    char *headerName;
    char *headerValue;
};
typedef struct header Header;
Header *createHeaderParser(char *buff)
{
    /*
     * NOTE: Caller must call freeHeaderParser after they
     * are done with it.
     *
     * Gets the buffer from  client, passes the first line
     * and parse all the `name: value\r\n` format into the
     * type Header and returns the pointer to first one.
     */
    int headerCount = DEFAULT_HEADER_COUNT;
    Header *headers = malloc(headerCount * sizeof(Header));
    exitNullPointer(headers);
    int i = 0;
    while (buff[i++] != '\n')
        ; // to pass first line
    int currHeaderPos = 0;
    while (1)
    {
        if (currHeaderPos >= headerCount)
        {
            headers = realloc(headers, sizeof(Header) * (headerCount + DEFAULT_HEADER_COUNT));
            exitNullPointer(headers);
            headerCount += DEFAULT_HEADER_COUNT;
            if (headerCount > MAX_HEADER_COUNT)
                exitLog("MAX_HEADER_COUNT exceed");
        }
        char **currHeaderName = &headers[currHeaderPos].headerName; // Just for convenience.
        char **currHeaderValue = &headers[currHeaderPos].headerValue;
        long int nameSize = DEFAULT_HEADER_FIELD_SIZE;
        long int valueSize = DEFAULT_HEADER_FIELD_SIZE;
        *currHeaderName = malloc(nameSize);
        *currHeaderValue = malloc(valueSize);
        exitNullPointer(*currHeaderName);
        exitNullPointer(*currHeaderValue);
        int pos = 0;
        while (buff[i] != ':')
        {
            if (nameSize >= MAX_HEADER_FIELD_SIZE)
                exitLog("MAX_HEADER_FIELD_SIZE exceed");
            if (pos >= nameSize)
            {
                nameSize *= 2; // double the size of field
                *currHeaderName = realloc(*currHeaderName, nameSize + 1);
                exitNullPointer(*currHeaderName);
            }
            (*currHeaderName)[pos++] = buff[i++];
        }
        (*currHeaderName)[pos] = '\0';
        i += 2; // pass space and :
        pos = 0;
        while (buff[i] != '\r')
        {
            if (valueSize >= MAX_HEADER_FIELD_SIZE)
                exitLog("MAX_HEADER_FIELD_SIZE exceed");
            if (pos >= valueSize)
            {
                valueSize *= 2; // double the size of field
                *currHeaderValue = realloc(*currHeaderValue, valueSize + 1);
                exitNullPointer(*currHeaderValue);
            }
            (*currHeaderValue)[pos++] = buff[i++];
        }
        (*currHeaderValue)[pos] = '\0';
        printf("%s,%s\n", *currHeaderName, *currHeaderValue); // to check if it has correctly parsed - won't be in final code. 
        currHeaderPos++;
        i += 2; // pass \r\n
        if (buff[i] == '\r')
        {
            headers[currHeaderPos].headerName = NULL; // end
            break;
        }
    }
    return headers;
}
void freeHeaderParser(Header *headers)
{
    for (int i = 0; headers[i].headerName != NULL; i++)
    {
        free(headers[i].headerName);
        free(headers[i].headerValue);
    }
    free(headers);
}

Answer 1

Answers to your questions

Is safe? (after all this is C)

No. Consider the very first while-loop in your code:

while (buff[i++] != '\n')
    ; // to pass first line`

What if there is no '\n' in the buffer? Make as little assumptions as possible about the input. Since you pass a string via a char *, the only thing you can assume is that it will be NUL-terminated, as there is no other way to deduce the length of the string.

Similarly, things like i += 2; // pass space and : are unsafe; don't assume there really is a space after the colon.

Also, "safe" is not very well defined. Is it safe that your function calls exit() on error? It will take down the whole program with it, which might not be desirable.

Does proper exit on failure? Are the ways that exitLog and exitNullPointer functions exit considered good practice?

Make sure you print errors to stderr, as stdout might already be used for other things, and it is less likely that stderr is redirected.

Prefer exit(EXIT_FAILURE) over a hardcoded number.

Has good implementation structure? Is returning a pointer to a struct and having a helper function to free the allocated memory common?

This is common. If you do have one function create memory, then it is very good practice to have another function for freeing that memory.

A completely different option would be to just return one header at a time, and keep track of where you are in buff. That way, your code would not have to deal with memory allocation at all.

Is having NULL in the header name to indicate end of headers a bad practice?

That's one way of doing it. Another option would be to return the number of elements in the array somehow.

Use strdup() and strndup()

Use strdup() and strndup() to copy strings instead of writing your own allocation routines. They are standard in c23, but even way before that it was part of almost all standard libraries.

Use const where appropriate

Make parameters const where possible. This will prevent errors where you accidentally write to input parameters, and it will sometimes also allow the compiler to better optimize code.

Reduce the amount of memory allocations

I already mentioned that you could probably solve this issue without any form of memory allocation. If you still want to parse all headers in one go, then consider that almost all of buff is going to be header names and header values, and just a tiny bit of whitespace and colons in between. So instead of copying each element into a separately allocated string, why not just duplicate buff, replace colons and the end of values with NUL bytes in that duplicate, and then just return an array pointers into that duplicate buffer?

Consider other ways to solve your actual problem

Do you really need a list of all the headers? If you are only want to search for a few specific headers, like Content-Length for example, then maybe you could make a function that search for a specific header and returns just the value?

char *findHeader(const char *buff, const char *header_name)
{
    …
}

Use empty lines to separate functions and code blocks

You don't have a single empty line in your program. That makes it harder to read. Try adding an empty line between every function, and also add empty lines inside functions to help separate different sections visually.

Answer 2

Strongly suggest you spend some time reading code before attempting to write your own.

I don't want to discourage you, but this code does not demonstrate a firm grasp of how data is processed efficiently.

Exercise: Kudos to you for correctly using "pointer to pointer" variables.
Now, revise the code to NOT use that extra level of indirection.
(Hint: the names in headers[currHeaderPos].headerValue are all pretty redundant and verbose. Try to use meaningful, but shorter, names for variables, especially those only visible within a function.)
Eg. hdrs[hdrPos].value[pos++] = buff[i++]; maybe??
Code must be correct, but should be lean and expressive; it's not "War and Peace".
Extra variables are rarely beneficial, and frequently homes for bugs.

---

Reinventing wheels

Hint: strtok( buff, ":\r\n" ); will chop the original string into the segments that you want without a lot of fuss. Store the returned pointers as 'name/value' pairs of pointers (odd/even using only one loop). Leave the strings in buff, or use pointer arithmetic to determine how much to allocate for each segment (and reassign the pointers to each piece...)

It's REALLY hard to see that every name or value allocation will be between 1000 (minimum!) and 20Mb in size... This, to me, seems outlandish...

Edit:
Other answers warn about strtok() and the possibility of const buffers.
One should always be aware of potential pitfalls of dealing with input.

FWIW: a very useful Standard Library function that is, too often, overlooked is strcspn() (and its sibling strspn()). You may wish to familiarise yourself with this pair and, perhaps, utilise one of them in your code.

Code is always better when it incorporates standard functions, rather than DIY constructs. Those library functions do what they're meant to do and are often faster and usually more robust than DIY code.

---

There's very little 'protection' against corrupt data, and every expectation of high quality data. The detection of reaching the end of the buffer to stop processing is extremely dodgy.

---

        if (currHeaderPos >= headerCount)
        {
            headers = realloc(headers, sizeof(Header) * (headerCount + DEFAULT_HEADER_COUNT));
            exitNullPointer(headers);
            headerCount += DEFAULT_HEADER_COUNT;
            if (headerCount > MAX_HEADER_COUNT)
                exitLog("MAX_HEADER_COUNT exceed");
        }

Why error exit AFTER the reallocation succeeded?
It appears you've not really worked out just how much/many items you want to peg as the arbitrary maximum your function will deal with. Lots of arbitrary values used, instead of measuring and adapting to what's been provided in the source data.
Hint: One pass over buff counting : characters would tell you how many name/value pairs to expect... No realloc() required...

---

Error messages go to stderr, not stdout.

---

        currHeaderPos++;
        i += 2; // pass \r\n
        if (buff[i] == '\r')
        {
            headers[currHeaderPos].headerName = NULL; // end
            break;
        }

Think VERY carefully about these lines...

The code allocated room for 30 name/value pairs. Say there were exactly 30 name/value pairs in the buff string... Where will that NULL be placed to mark the end of the used pointers?

The same "filled to capacity, but tried to sneak in a '\0' at the end" applies to both the string copying regions of this code.

Exercise: Revise the outer loop to use do/while() instead of while(1).
What almost trivial changes are needed?
Does this simplify the logic a tiny bit?

---

DRY: Don't Repeat Yourself

There are two large-ish while() loops in the middle of the big function.

These lines should have been factored out as a single 'worker function' that allocates and fills the buffer until the 'magic' marking character is reached.

Strive to avoid duplication of functionality with copy/paste/adapt coding. Write capable functions that work from a few parameters. Don't repeat yourself.

Reflecting on those pointer-to-pointer variables, using a well considered function and its parameters, there would be less code and the 'contained' code that remains could use much simpler indirections.

KISS means "Keep It Simple Simon" where 'Simon' is usually another word...

---

You're right. This is the C language, meaning the coder can't just be slack and relaxed believing that the language has lots of guardrails and padded corners that protect sloppy coders from themselves. Attention and discipline is a prerequisite.

Peter Parker's uncle Ben said it: "With great power comes great responsibility."

---

Consider:

void *allocOrDie( void *ptr, size_t size )
{
    ptr = realloc( ptr, size );

    if( ptr )
        return ptr;

    fprintf( stderr, "Malloc/Realloc failed.\n" );
    exit( EXIT_FAILURE );
}

Header *createHeaderParser( char *buff )
{
    ...
    Header *headers = allocOrDie( NULL, sizeof *headers * headerCount );
    ...
        if( currHeaderPos >= headerCount )
        {
            headerCount += DEFAULT_HEADER_COUNT;
            if( headerCount > MAX_HEADER_COUNT )
                exitLog("MAX_HEADER_COUNT exceed");

            headers = allocOrDie( headers, sizeof *headers * headerCount );
            ...

The above is quite crude and not optimal. Intended to show the value of READING over WRITING for beginners. In the documentation you'll find that realloc() passed a NULL pointer acts just like malloc(). Yes, it's fun to code some running programs as exercises. When starting out, it's more advantageous to study examples to see how experienced people have solved different issues.

As said, shown here is a crude partial transition toward less code. Plan it out and think it through... Delay allocation until AFTER the code has determined there's something it needs to store in heap. Done well, instead of malloc() plus realloc(), only ONE call to realloc() is required (inside the loop) that grows from a start of 0 elements.

And don't overlook the requirement for the extra "end of list" NULL marker.

Exercise: Write allocOrBounce() that does not terminate, but somehow signals the caller that allocation failed and the caller can try to recover somehow...

---

To ponder...

Thunderbird email client stores emails as plain text, then (re-)generates and uses a 'companion' file of its own indexing for speed. (It's a simple, flexible form of a database full of 'blobs'.) I grew tired of waiting for backups when I could see that a one word email (e.g. a reply payload of "Okay!") was prefaced by 5-20Kb of 'transport layer logging statements', so I wrote a filter that thinned those down to what was useful to Thunderbird and to me.

This HTTP problem feels similar.

Just wanted to note that

    char *endOfHdr = strstr( blob, "\r\n\r\n" );

gave me the 'pivot point' between what was header (preceding) and what was payload (following). I could then go on to deal with both portions as two completely separate sections.

Advice to OP: Find and read the man pages for all of the functions available in C's Standard Library. Some you'll use frequently, some only on rare occasions. Re-visit those man pages every few months to refresh your memory of what's tried-and-true and available.

One of the reasons C has survived for over 50 years is that, with its library functions, it lives at the Goldilocks point on the spectrum of languages; not too much like a 'kitchen sink' and not too sparse or barren or wanting. IOW "just right".

Answer 3

Why NOT strtok (updated)

1. Overwrites delimiters (mutates buffer)

char buffer[] = "Host: example.com\r\n";
char *name = strtok(buffer, ":");  // buffer now "Host\0 example.com\r\n"

// String view past "Host" is now split at the inserted NUL; 
// original text no longer intact as one string.

If you need the original contiguous text later, you must have saved a copy (pointer+len) before calling strtok.

2. Not safely reentrant / thread-friendly

strtok uses internal static state for continuation calls (strtok(NULL, ...)). Interleaving uses across threads (or even in the same thread across nested parsers) is not portable and can corrupt tokenization. Use manual parsing, strtok_r (POSIX), or strtok_s (Annex K, rarely available) if you must.

// Thread 1: strtok(buf1, ":");
// Thread 2: strtok(buf2, ":");  

// Interference possible if either parser does continuation (NULL first arg).

3. Collapses delimiters → empty vs missing indistinguishable

// Use one delimiter *set* to show the problem clearly.
char line1[] = "X-Empty-Header: \r\n";  // valid empty value (space = OWS)
char *name = strtok(line1, ": \r\n");   // "X-Empty-Header"
char *val  = strtok(NULL, ": \r\n");    // NULL (empty field skipped)

char line2[] = "X-Truncated:";          // malformed: no value or CRLF
name = strtok(line2, ": \r\n");
val  = strtok(NULL, ": \r\n");          // NULL (no token)

Because strtok collapses runs of delimiters and never produces empty tokens, both a syntactically valid empty field and a truncated/malformed line end up looking the same (NULL).

4. Splits at all delimiters in the set

Delimiters form a set, not a sequence. Every : ends a token.

char line[] = "Authorization: Bearer abc:def:123\r\n";
strtok(line, ":");   // "Authorization"
strtok(NULL, ":");   // " Bearer abc"
strtok(NULL, ":");   // "def"
strtok(NULL, ":");   // "123\r\n"

You lose the intended value boundary.

5. Requires writable memory

strtok writes '\0' into the string. Passing a string literal (often in read-only storage) is undefined behavior (commonly crashes).

// read-only storage (string literal)
const char *request = "GET / HTTP/1.1\r\n";
// UB: writes into read-only storage; may crash 
char *line = strtok((char *)request, "\r\n"); 
// Some compilers warn on discarded 'const'; cast silences warning but not UB.
// Better solution: copy string literal to a mutable buffer first.

Why NOT strdup (updated)

1. Not standard before C23 (POSIX extension)

strdup (and strndup) were POSIX/GNU extensions until C23. On older strictly conforming C environments you must enable the extension (e.g., #define _POSIX_C_SOURCE 200809L before includes) or provide your own implementation. strdup is now standard in C23.

2. Whole-buffer duplication can waste memory

If you only need a few small substrings from a large request, duplicating the entire buffer doubles peak memory usage for no benefit.

// Inefficient: strdup entire buffer, then parse
char request[] = "GET / HTTP/1.1\r\nHost: example.com\r\nUser-Agent: curl\r\n\r\n";
char *dup = strdup(request);  // ~60 bytes allocated
// Parse headers, maybe malloc each field... 
// Total: 60 bytes (dup) + field copies
free(dup);

// vs. Efficient: parse and copy only what's needed (see alternatives below)

3. Doesn't inherently reduce allocations

Whether strdup helps depends on usage:

• If you strdup once and then point into that buffer (no per-field malloc), it can reduce allocations.
• If you strdup whole buffer and still malloc copies of each token, you've added overhead. Measure.

Better alternatives

Simple: strchr + explicit copy

Avoid strncpy (zero-pads; may not NUL-terminate if truncated). Use memcpy + manual NUL.

const char *colon = strchr(line, ':');
if (colon) {
    size_t name_len = (size_t)(colon - line);
    memcpy(name, line, name_len);
    name[name_len] = '\0';
}

Efficient approach (length-bounded, non-destructive, OWS-aware)

// Parse "Name: [OWS]Value\r\n" without altering the original buffer.
// OWS = optional whitespace (space or tab) per HTTP/1.x.
// Allocates name and value in a single malloc for cache locality.
// Note: Does not handle obsolete header line folding (RFC 7230 deprecated).
// Returns 0 on success; -1 if no colon or empty name; -2 if alloc failure.
int parse_header_line(const char *line, char **out_name, char **out_value) {
    const char *colon = strchr(line, ':');
    if (!colon || colon == line) return -1;  // No colon or empty name

    size_t name_len = (size_t)(colon - line);

    // Skip optional whitespace after colon
    const char *value_start = colon + 1;
    while (*value_start == ' ' || *value_start == '\t') value_start++;

    // Find end of line (CR or LF); tolerate missing CRLF at buffer end
    const char *p = value_start;
    while (*p && *p != '\r' && *p != '\n') p++;
    size_t value_len = (size_t)(p - value_start);

    // Single allocation for both strings (name + NUL + value + NUL)
    size_t total = name_len + 1 + value_len + 1;
    if (name_len > SIZE_MAX - 2 - value_len) return -2; // Prevent integer overflow
    
    char *buffer = malloc(total);
    if (!buffer) return -2;

    char *name  = buffer;
    char *value = buffer + name_len + 1;

    memcpy(name, line, name_len);
    name[name_len] = '\0';

    memcpy(value, value_start, value_len);
    value[value_len] = '\0';

    *out_name  = name;
    *out_value = value;
    return 0;
}

Memory usage comparison (approximate)

Assume avg header name + value totals ~50 bytes incl NULs.

Approach	Per-header alloc	For 50 headers	Notes
Fixed 1000+1000	~2000 B	~100 KB	Large static buffers.
Full-buffer strdup + per-field copies	4 KB + ~2.5 KB	~6.5 KB	strdup whole buffer + field mallocs.
Exact alloc (above)	~50 B	~2.5 KB	Single malloc per header.
Single strdup, no field allocs	n/a	4 KB	Works if consumer accepts non-NUL slices.
Slices only	0 B	0 B	Zero-copy parsing

Critical performance optimisation: Parse into slices; duplicate only when storing.

Slice helper if you don't need owned strings yet

struct slice { const char *ptr; size_t len; };
// Parse into slices; duplicate only when storing.
struct slice { const char *ptr; size_t len; };

int parse_header_slice(const char *line, struct slice *name, struct slice *value) {
    const char *colon = strchr(line, ':');
    if (!colon || colon == line) return -1;
    
    name->ptr = line;
    name->len = (size_t)(colon - line);
    
    const char *value_start = colon + 1;
    while (*value_start == ' ' || *value_start == '\t') value_start++;
    
    const char *p = value_start;
    while (*p && *p != '\r' && *p != '\n') p++;
    
    value->ptr = value_start;
    value->len = (size_t)(p - value_start);
    return 0;
}

Also I encourage you to see how it's done in:

• Nginx

• Varnish

• httplite