tokenizer Class Reference

Abstract baseclass for the tokenizer. More...

#include <tokenizer.hpp>

Collaboration diagram for tokenizer:

Collaboration graph
[legend]

List of all members.

Public Member Functions

 tokenizer (std::istream &in)
 ~tokenizer ()
const tokennext_token ()
const tokencurrent_token () const
std::string get_line () const
std::string & textdomain ()

Protected Member Functions

 tokenizer ()
void next_char ()
void next_char_fast ()
int peek_char () const

Protected Attributes

int current_
size_t lineno_

Private Member Functions

bool is_space (const int c) const
bool is_alnum (const int c) const
void skip_comment ()

Private Attributes

std::string textdomain_
std::string file_
size_t tokenstart_lineno_
token token_
std::istream & in_


Detailed Description

Abstract baseclass for the tokenizer.

Definition at line 58 of file tokenizer.hpp.


Constructor & Destructor Documentation

tokenizer::tokenizer ( std::istream &  in  ) 

Definition at line 27 of file tokenizer.cpp.

References next_char_fast().

tokenizer::~tokenizer (  )  [inline]

Definition at line 62 of file tokenizer.hpp.

tokenizer::tokenizer (  )  [protected]


Member Function Documentation

const token & tokenizer::next_token (  ) 

const token & tokenizer::current_token (  )  const

Definition at line 194 of file tokenizer.cpp.

References token_.

std::string tokenizer::get_line (  )  const

Definition at line 211 of file tokenizer.cpp.

References file_, and tokenstart_lineno_.

std::string & tokenizer::textdomain (  ) 

Definition at line 218 of file tokenizer.cpp.

References textdomain_.

void tokenizer::next_char (  )  [inline, protected]

Definition at line 74 of file tokenizer.hpp.

References current_, lineno_, next_char_fast(), and UNLIKELY.

Referenced by next_token().

void tokenizer::next_char_fast (  )  [inline, protected]

Definition at line 81 of file tokenizer.hpp.

References current_, in_, LIKELY, and UNLIKELY.

Referenced by next_char(), next_token(), skip_comment(), and tokenizer().

int tokenizer::peek_char (  )  const [inline, protected]

Definition at line 113 of file tokenizer.hpp.

References in_.

Referenced by next_token().

bool tokenizer::is_space ( const int  c  )  const [private]

Definition at line 200 of file tokenizer.cpp.

Referenced by next_token().

bool tokenizer::is_alnum ( const int  c  )  const [private]

Definition at line 205 of file tokenizer.cpp.

Referenced by next_token().

void tokenizer::skip_comment (  )  [private]

Definition at line 39 of file tokenizer.cpp.

References current_, file_, lexical_cast(), lineno_, next_char_fast(), and textdomain_.

Referenced by next_token().


Member Data Documentation

int tokenizer::current_ [protected]

Definition at line 71 of file tokenizer.hpp.

Referenced by next_char(), next_char_fast(), next_token(), and skip_comment().

size_t tokenizer::lineno_ [protected]

Definition at line 72 of file tokenizer.hpp.

Referenced by next_char(), next_token(), and skip_comment().

std::string tokenizer::textdomain_ [private]

Definition at line 123 of file tokenizer.hpp.

Referenced by skip_comment(), and textdomain().

std::string tokenizer::file_ [private]

Definition at line 124 of file tokenizer.hpp.

Referenced by get_line(), and skip_comment().

Definition at line 125 of file tokenizer.hpp.

Referenced by get_line(), and next_token().

Definition at line 126 of file tokenizer.hpp.

Referenced by current_token(), and next_token().

std::istream& tokenizer::in_ [private]

Definition at line 127 of file tokenizer.hpp.

Referenced by next_char_fast(), and peek_char().


The documentation for this class was generated from the following files:

Generated by doxygen 1.5.5 on 23 May 2008 for The Battle for Wesnoth
Gna! | Forum | Wiki | CIA | devdocs