Module Basillsp.Tokens

The lexer gives us character token ranges (start_pos: int, end_pos: int) as an absolute offset from the beginning of the document, while LSP works over (line, column) positions.

This module handles converting between and query across position representations.

LineCol.t represents a line-column character position, while Token.t represents a (LineCol.t, size) token character interval.

We use an IntMap.t inverse map from <begin line character offset> -> <line number> to convert between these.

type linebreaks = int Common.Util.IntMap.t

map from character beginning a line to the line number it begins.

val linebreaks : string -> linebreaks
val get_begin_line : linebreaks -> int -> Common.Util.IntMap.key
val get_linenum : linebreaks -> int -> int
module LineCol : sig ... end
module Token : sig ... end
module TokenMap : sig ... end
val loc_of_char_pos : linebreaks -> int -> LineCol.t
val range_of_position : linebreaks -> Stdlib.Lexing.position -> Stdlib.Lexing.position -> Linol_lsp.Types.Range.t
val token_of_char_range : linebreaks -> int -> int -> Token.t
val token_of_lexer_token : linebreaks -> ((int * int) * string) -> Token.t
val find_token_opt : 'a TokenMap.t -> LineCol.t -> 'a option