Index of values

__private__next_int lexbuf extracts the next code point from the lexer buffer and increments to current position.

backtrack lexbuf returns the value stored in the internal slot of the buffer, and performs backtracking (the current position is set to the value of the backtrack position).

Create a generic lexer buffer.

Works as Utf16.from_gen with an in_channel.

Create a lexbuf from a UTF-8 encoded input channel.

Create a lexbuf from a Latin1 encoded input channel.

Utf16.from_gen s opt_bo creates a lexbuf from an UTF-16 encoded stream.

Create a lexbuf from a UTF-8 encoded stream.

Create a lexbuf from a Latin1 encoded stream (ie a stream of Unicode code points in the range 0..255)

Create a lexbuf from a stream of Unicode code points.

Create a lexbuf from an array of Unicode code points.

Works as Utf16.from_gen with a string.

Create a lexbuf from a UTF-8 encoded string.

Create a lexbuf from a Latin1 encoded string.

Create a lexbuf from an array of Unicode code points.

utf16_lexeme lb bo bom as Sedlexing.lexeme with a result encoded in UTF-16 in byte_order bo and starting with a BOM if bom = true.

As Sedlexing.lexeme with a result encoded in UTF-8.

As Sedlexing.lexeme with a result encoded in Latin1.

Sedlexing.lexeme lexbuf returns the string matched by the regular expression as an array of Unicode code point.

As Sedlexing.lexeme_char with a result encoded in Latin1.

Sedlexing.lexeme_char lexbuf pos returns code point number pos in the matched string.

Sedlexing.lexeme_end lexbuf returns the offset in the input stream of the character following the last code point of the matched string.

Sedlexing.lexeme_length lexbuf returns the difference (Sedlexing.lexeme_end lexbuf) - (Sedlexing.lexeme_start lexbuf), that is, the length (in code points) of the matched string.

Sedlexing.lexeme_start lexbuf returns the offset in the input stream of the first code point of the matched string.

Sedlexing.lexing_positions lexbuf returns the start and end positions of the current token, using a record of type Lexing.position.

Sedlexing.loc lexbuf returns the pair (Sedlexing.lexeme_start lexbuf,Sedlexing.lexeme_end lexbuf).

mark lexbuf i stores the integer i in the internal slot.

Sedlexing.new_line lexbuf increments the line count and sets the beginning of line to the current position, as though a newline character had been encountered in the input.

next lexbuf extracts the next code point from the lexer buffer and increments to current position.

Sedlexing.rollback lexbuf puts lexbuf back in its configuration before the last lexeme was matched.

set_filename lexbuf file sets the filename to file in lexbuf.

set the initial tracked input position for lexbuf.

start t informs the lexer buffer that any code points until the current position can be discarded.

sub_lexeme lb pos len bo bom as Sedlexing.sub_lexeme with a result encoded in UTF-16 with byte order bo and starting with a BOM if bom=true

As Sedlexing.sub_lexeme with a result encoded in UTF-8.

As Sedlexing.sub_lexeme with a result encoded in Latin1.

Sedlexing.sub_lexeme lexbuf pos len returns a substring of the string matched by the regular expression as an array of Unicode code point.

with_tokenizer tokenizer lexbuf given a lexer and a lexbuf, returns a generator of tokens annotated with positions.