🔗 dashbitco/lazy_html: Efficient parsing and querying of HTML documents

Efficient parsing and querying of HTML documents.

LazyHTML is designed around lazy HTML documents. Documents are parsed and kept natively in memory for as long as possible. Query selectors are executed in native code for performance and adheres to browser standards. Under the hood, LazyHTML uses Lexbor, a fast, dependency-free and comprehensive HTML engine, written entirely in C.

LazyHTML works with a flat list of nodes and all operations are batched by default, as shown below:

lazy_html =
  LazyHTML.from_fragment("""
  <div>
    <a href="https://elixir-lang.org">Elixir</a>
    <a href="https://www.erlang.org">Erlang</a>
  </div>\
  """)
#=> #LazyHTML<
#=>   1 node
#=>
#=>   #1
#=>   <div>
#=>     <a href="https://elixir-lang.org">Elixir</a>
#=>     <a href="https://www.erlang.org">Erlang</a>
#=>   </div>
#=> >

hyperlinks = LazyHTML.query(lazy_html, "a")
#=> #LazyHTML<
#=>   2 nodes (from selector)
#=>
#=>   #1
#=>   <a href="https://elixir-lang.org">Elixir</a>
#=>
#=>   #2
#=>   <a href="https://www.erlang.org">Erlang</a>
#=> >

LazyHTML.attribute(hyperlinks, "href")
#=> ["https://elixir-lang.org", "https://www.erlang.org"]

LazyHTML also provides several high-level conveniences:

• an Inspect implementation to pretty-print nodes
• an Access implementation to run CSS selectors
• an Enumerable implementation to traverse them

For example:

lazy_html = LazyHTML.from_fragment(~S|<p><strong>Hello</strong>, <em>world</em>!</p>|)
#=> #LazyHTML<
#=>   1 node
#=>
#=>   #1
#=>   <p><strong>Hello</strong>, <em>world</em>!</p>
#=> >

lazy_html["strong, em"]
#=> #LazyHTML<
#=>   2 nodes (from selector)
#=>
#=>   #1
#=>   <strong>Hello</strong>
#=>
#=>   #2
#=>   <em>world</em>
#=> >

LazyHTML.text(lazy_html)
#=> "Hello, world!"

Enum.map(lazy_html["strong, em"], &LazyHTML.text/1)
#=> ["Hello", "world"]

If needed, the lazy nodes can be converted into an Elixir tree data structure, and vice-versa.

lazy_html = LazyHTML.from_fragment("<p><strong>Hello</strong>, <em>world</em>!</p>")
#=> #LazyHTML<
#=>   1 node
#=>
#=>   #1
#=>   <p><strong>Hello</strong>, <em>world</em>!</p>
#=> >

tree = LazyHTML.to_tree(lazy_html)
#=> [{"p", [], [{"strong", [], ["Hello"]}, ", ", {"em", [], ["world"]}, "!"]}]

LazyHTML.from_tree(tree)
#=> #LazyHTML<
#=>   1 node

#=>   #1
#=>   <p><strong>Hello</strong>, <em>world</em>!</p>
#=> >