Long time ago, I posted a text scanner on flipcode (link). Couple of years later, I posted an update with all kinds of snazzy new features, a namespace, and what not.

Today, I’m reposting, with a hardcore regression to C.

-> TextScanner.h
-> TextScanner.c

This version has fewer fancy features, fewer lines of code, and more straight up dealing with text. It has the functions that proved most useful and robust over the years. It has no internal state, making it first of all threadsafe, and secondly data-oriented.

Here’s a preview of the public API. Give it a try if you need this sort of thing. Feedback welcome!


// Get Token
EXTERNC char const* tsGetToken (char const* pCurr, char const* pEnd, char delim, char const** resultStringBegin, uint32_t* stringLength);
EXTERNC char const* tsGetTokenWSDelimited (char const* pCurr, char const* pEnd, char const** resultStringBegin, uint32_t* stringLength);
EXTERNC char const* tsGetTokenAlphaNumeric (char const* pCurr, char const* pEnd, char const** resultStringBegin, uint32_t* stringLength);
EXTERNC char const* tsGetNameSpacedTokenAlphaNumeric(char const* pCurr, char const* pEnd, char namespaceChar, char const** resultStringBegin, uint32_t* stringLength);

// Get Value
EXTERNC char const* tsGetString (char const* pCurr, char const* pEnd, char const** resultStringBegin, uint32_t* stringLength);
EXTERNC char const* tsGetInt16 (char const* pCurr, char const* pEnd, int16_t* result);
EXTERNC char const* tsGetInt32 (char const* pCurr, char const* pEnd, int32_t* result);
EXTERNC char const* tsGetUInt32 (char const* pCurr, char const* pEnd, uint32_t* result);
EXTERNC char const* tsGetHex (char const* pCurr, char const* pEnd, uint32_t* result);
EXTERNC char const* tsGetFloat (char const* pcurr, char const* pEnd, float* result);

EXTERNC char const* tsScanForCharacter (char const* pCurr, char const* pEnd, char delim);
EXTERNC char const* tsScanBackwardsForCharacter (char const* pCurr, char const* pEnd, char delim);
EXTERNC char const* tsScanForWhiteSpace (char const* pCurr, char const* pEnd);
EXTERNC char const* tsScanBackwardsForWhiteSpace (char const* pCurr, char const* pStart);
EXTERNC char const* tsScanForNonWhiteSpace (char const* pCurr, char const* pEnd);
EXTERNC char const* tsScanForTrailingNonWhiteSpace (char const* pCurr, char const* pEnd);
EXTERNC char const* tsScanForQuote (char const* pCurr, char const* pEnd);
EXTERNC char const* tsScanForEndOfLine (char const* pCurr, char const* pEnd);
EXTERNC char const* tsScanForLastCharacterOnLine (char const* pCurr, char const* pEnd);
EXTERNC char const* tsScanForBeginningOfNextLine (char const* pCurr, char const* pEnd);
EXTERNC char const* tsScanPastCPPComments (char const* pCurr, char const* pEnd);
EXTERNC char const* tsSkipCommentsAndWhitespace (char const* pCurr, char const*const pEnd);

EXTERNC _Bool tsIsWhiteSpace(char test);
EXTERNC _Bool tsIsNumeric (char test);
EXTERNC _Bool tsIsAlpha (char test);
EXTERNC _Bool tsIsIn (const char* testString, char test);