A structured lesson workspace with readable content, hands-on examples, and a clean path to completion.
strtok splits a string into tokens separated by delimiter characters. The first call takes the string and the delimiters. Subsequent calls take NULL and the same delimiters, continuing where the previous call left off. It modifies the input string by replacing delimiters with null terminators. It stores its position in a static internal variable. That static state makes it non-reentrant and unsafe in multithreaded code. POSIX provides strtok_r to fix this. The C standard does not.
#include <stdio.h>#include <string.h>int main(void) {char csv[] = "apple,banana,cherry,date";char *token = strtok(csv, ",");while (token != NULL) {printf("Token: \"%s\"\n", token);token = strtok(NULL, ",");}printf("\nOriginal buffer after strtok: \"%s\"\n", csv);return 0;}
The original buffer now contains only "apple" because strtok replaced the first comma with \0. The remaining tokens are accessible through the returned pointers, which point into the modified buffer. If you need the original string intact, copy it before tokenizing. strtok is destructive.
#include <stdio.h>#include <string.h>int main(void) {char log[] = "2026-08-02 10:15:32 ERROR Disk full";char *token = strtok(log, " ");while (token) {printf("[%s]\n", token);token = strtok(NULL, " ");}return 0;}
The two spaces between the timestamp and ERROR were treated as a single delimiter. strtok skips consecutive delimiters. There is no empty token between them. This is convenient for whitespace splitting but wrong if you need to preserve empty fields, as in CSV where "a,,b" has an empty middle field. For that, use a manual parser or strpbrk.
strtok stores its state in a static variable. If two threads call strtok concurrently, they corrupt each other's position. The second thread's strtok(NULL, ...) call continues from where the first thread left off, not from its own string. The fix on POSIX is strtok_r, which takes an extra char** parameter to hold the state per-call.
#include <stdio.h>#include <string.h>int main(void) {char data[] = "one:two:three";char *saveptr;char *token = strtok_r(data, ":", &saveptr);while (token) {printf("Token: %s\n", token);token = strtok_r(NULL, ":", &saveptr);}return 0;}
Never use strtok in multithreaded code or in library functions that might be called concurrently. Use strtok_r (POSIX) or strtok_s (C11 Annex K / Windows). If neither is available, write a manual tokenizer with strpbrk. The static state in strtok is a concurrency bug waiting to happen.
strtok is convenient for quick scripts and single-threaded utilities. In production code, especially concurrent code, prefer strtok_r or a manual parser. The static state is a design artifact from the 1970s that has caused decades of bugs. Next: strcspn, which computes the length of a prefix that avoids a set of characters.
Finish the lesson once you have worked through the material. This awards ★ 30 XP.