Packages

  • package root
    Definition Classes
    root
  • package io
    Definition Classes
    root
  • package dylemma
    Definition Classes
    io
  • package spac

    SPaC (short for "Streaming Parser Combinators") is a library for building stream consumers in a declarative style, specialized for tree-like data types like XML and JSON.

    SPaC (short for "Streaming Parser Combinators") is a library for building stream consumers in a declarative style, specialized for tree-like data types like XML and JSON.

    Many utilities for handling XML and JSON data involve parsing the entire "document" to some DOM model, then inspecting and transforming that model to extract information. The downside to these utilities is that when the document is very large, the DOM may not fit in memory. The workaround for this type of problem is to treat the document as a stream of "events", e.g. "StartElement" and "EndElement" for XML, or "StartObject" and "EndObject" for JSON. The downside to this workaround is that writing code to handle these streams can be complicated and error-prone, especially when the DOM is complicated.

    SPaC's goal is to drastically simplify the process of creating code to handle these streams.

    This package contains the "core" SPaC traits; Parser, Transformer, Splitter, and ContextMatcher.

    See the xml and json subpackages (provided by the xml-spac and json-spac libraries respectively) for specific utilities related to handling XML and JSON event streams.

    Definition Classes
    dylemma
  • package interop
    Definition Classes
    spac
  • package json

    This package provides extensions to the core "spac" library which allow for the handling of JSON data.

    This package provides extensions to the core "spac" library which allow for the handling of JSON data.

    Rather than creating explicit classes that extend Parser, Transformer, and Splitter, this package provides type aliases and implicit extensions. For example, JsonParser[A] is just a type alias for Parser[JsonEvent, A], and JsonParser is just a call to Parser[JsonEvent].

    Implicit JsonParsers are available for each of the JSON primitive types:

    • string
    • number (expressed as Int, Long, Float, or Double)
    • boolean
    • null (expressed as None.type)

    Helpers are available for parsing JSON arrays and objects:

    • JsonParser.listOf[A] to parse an array where each value is an A
    • JsonParser.objectOf[A] to parse an object where the value for each field an A
    • JsonParser.objectOfNullable[A] to parse an object where the value for each field is either null or an A, filtering out the nulls
    • JsonParser.fieldOf[A](fieldName) to parse a specific field from an object

    A DSL for creating json-specific ContextMatchers is provided to make it more convenient to call Splitter.json. For example:

    Splitter.json("foo" \ "bar").as[String].parseFirst

    Can be used to capture rootJson.foo.bar as a String in

    {
      "foo": {
        "bar": "hello"
      }
    }

    To "split" values inside arrays, index-related context matchers are available, e.g.

    Splitter.json("foo" \ anyIndex).as[Int].parseToList

    Can be used to capture each of the numbers in the "foo" array in

    {
      "foo": [1, 2, 3]
    }

    A note about JsonEvents in spac: JSON doesn't have any explicit markers for when a field ends, or when an array index starts or ends; those context changes are essentially inferred by the presence of some other event. For example, instead of a "field end" event, typically there will be either a new "field start" or a token representing the end of the current object. With spac, splitters and context matchers generally operate under the assumption that a "stack push" event (like a field start) will eventually be followed by a corresponding "stack pop" event (i.e. field end).

    To allow for this, these "inferred" events (FieldEnd, IndexStart, IndexEnd) are explicitly represented as JsonEvents in the stream being parsed. Keep this in mind when creating JSON ContextMatchers:

    • field-related matchers will match a stack like case ObjectStart :: FieldStart(_) :: _
    • index-related matchers will match a stack like case ArrayStart :: IndexStart(_) :: _
    Definition Classes
    spac
  • Fs2DataSource
  • JacksonSource
  • JsonEvent
  • JsonParserApplyOps
  • JsonSplitterApplyOps
  • JsonSplitterOps
  • JsonStackElem
  • JsonStackPop
  • JsonValueEvent
  • package xml

    This package provides extensions to the core "spac" library which allow for the handling of XML data.

    This package provides extensions to the core "spac" library which allow for the handling of XML data.

    Rather than creating explicit classes that extend Parser, Transformer, and Splitter, this package provides type aliases and implicit extensions. For example, XmlParser[A] is just a type alias for Parser[XmlEvent, A], and XmlParser is just a call to Parser[XmlEvent].

    Three main Parser methods are added to Parser[XmlEvent] via the XmlParserApplyOps implicit class:

    • XmlParser.forText - for capturing raw text
    • XmlParser.attr - for capturing mandatory attributes from elements
    • XmlParser.attrOpt - for capturing optional attributes from elements

    One main Splitter constructor method is added to Splitter via the XmlSplitterApplyOps implicit class:

    • Splitter.xml - for creating splitters based on an inspection of an "element stack"

    Three main Splitter member methods are added to Splitter[XmlEvent, C] via the XmlSplitterOps implicit class:

    • .attr - alias for .joinBy(XmlParser.attr(...))
    • .attrOpt - alias for .joinBy(XmlParser.attrOpt(...))
    • .text - alias for .joinBy(XmlParser.forText)

    A DSL for creating xml-specific ContextMatchers is provided to make it more convenient to call Splitter.xml. For example:

    Splitter.xml("things" \ "thing").attr("foo").parseToList

    Can be used to capture a list of the "foo" attributes in the <thing> elements in

    <things>
       <thing foo="hello" />
       <thing foo="Goodbye">
          <extra>junk</extra>
       </thing>
    </thing>
    Definition Classes
    spac

package json

This package provides extensions to the core "spac" library which allow for the handling of JSON data.

Rather than creating explicit classes that extend Parser, Transformer, and Splitter, this package provides type aliases and implicit extensions. For example, JsonParser[A] is just a type alias for Parser[JsonEvent, A], and JsonParser is just a call to Parser[JsonEvent].

Implicit JsonParsers are available for each of the JSON primitive types:

  • string
  • number (expressed as Int, Long, Float, or Double)
  • boolean
  • null (expressed as None.type)

Helpers are available for parsing JSON arrays and objects:

  • JsonParser.listOf[A] to parse an array where each value is an A
  • JsonParser.objectOf[A] to parse an object where the value for each field an A
  • JsonParser.objectOfNullable[A] to parse an object where the value for each field is either null or an A, filtering out the nulls
  • JsonParser.fieldOf[A](fieldName) to parse a specific field from an object

A DSL for creating json-specific ContextMatchers is provided to make it more convenient to call Splitter.json. For example:

Splitter.json("foo" \ "bar").as[String].parseFirst

Can be used to capture rootJson.foo.bar as a String in

{
  "foo": {
    "bar": "hello"
  }
}

To "split" values inside arrays, index-related context matchers are available, e.g.

Splitter.json("foo" \ anyIndex).as[Int].parseToList

Can be used to capture each of the numbers in the "foo" array in

{
  "foo": [1, 2, 3]
}

A note about JsonEvents in spac: JSON doesn't have any explicit markers for when a field ends, or when an array index starts or ends; those context changes are essentially inferred by the presence of some other event. For example, instead of a "field end" event, typically there will be either a new "field start" or a token representing the end of the current object. With spac, splitters and context matchers generally operate under the assumption that a "stack push" event (like a field start) will eventually be followed by a corresponding "stack pop" event (i.e. field end).

To allow for this, these "inferred" events (FieldEnd, IndexStart, IndexEnd) are explicitly represented as JsonEvents in the stream being parsed. Keep this in mind when creating JSON ContextMatchers:

  • field-related matchers will match a stack like case ObjectStart :: FieldStart(_) :: _
  • index-related matchers will match a stack like case ArrayStart :: IndexStart(_) :: _
Source
package.scala
Linear Supertypes
AnyRef, Any
Ordering
  1. Grouped
  2. Alphabetic
  3. By Inheritance
Inherited
  1. json
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Type Members

  1. type JsonContextMatcher[+C] = ContextMatcher[JsonStackElem, C]

  2. sealed trait JsonEvent extends HasLocation

    ADT for tokens in a JSON stream.

  3. type JsonParser[+Out] = Parser[JsonEvent, Out]

    Type alias for a Parser whose input type is JsonEvent

  4. implicit final class JsonParserApplyOps extends AnyVal

    Provides JSON-specific Parser constructor methods to the JsonParser object, for example JsonParser.fieldOf.

    Provides JSON-specific Parser constructor methods to the JsonParser object, for example JsonParser.fieldOf.

    Technically JsonParser is not a companion object, it is a partially-applied version of the Parser companion object which binds the input type to JsonEvent, so "companion methods" must instead be added as extension methods.

  5. type JsonSplitter[+C] = Splitter[JsonEvent, C]

    Type alias for a Splitter whose input type is JsonEvent

  6. implicit final class JsonSplitterApplyOps extends AnyVal

    Adds Splitter.json, for constructing json context matcher-based JsonSplitters

  7. implicit final class JsonSplitterOps[C] extends AnyVal

    Adds splitter.asNullable[A], for handling possibly-null values in a JSON substream

  8. sealed trait JsonStackElem extends JsonEvent

    Subset of JsonEvents that constitute a "context stack push".

  9. sealed trait JsonStackPop extends JsonEvent

    Subset of JsonEvents that constitute a "context stack pop".

  10. type JsonTransformer[+Out] = Transformer[JsonEvent, Out]

    Type alias for a Transformer whose input type is JsonEvent.

  11. sealed trait JsonValueEvent extends JsonEvent

    Subset of JsonEvents that represent a primitive values

Value Members

  1. val JsonParser: ParserApplyWithBoundInput[JsonEvent]

    Like the Parser companion object, but only for creating Parsers whose input type is JsonEvent.

    Like the Parser companion object, but only for creating Parsers whose input type is JsonEvent.

    See also

    JsonParserApplyOps

  2. val JsonSplitter: SplitterApplyWithBoundInput[JsonEvent]

    Like the Splitter companion object, but only for creating Splitters whose input type is JsonEvent.

  3. val JsonTransformer: TransformerApplyWithBoundInput[JsonEvent]

    Like the Transformer companion object, but only for creating Transformers whose input type is JsonEvent.

  4. def anyField: JsonContextMatcher[String]

    Context matcher that matches events with any JSON object field, capturing the name of the field as context.

    Context matcher that matches events with any JSON object field, capturing the name of the field as context. Specifically this matcher looks for an ObjectStart followed by a FieldStart(name), for any name.

  5. def anyIndex: JsonContextMatcher[Int]

    Context matcher that matches events within a JSON array, at any index.

    Context matcher that matches events within a JSON array, at any index. Specifically this matcher looks for an ArrayStart followed by an IndexStart(_) in the JSON context stack. If the match succeeds, the i index will be captured as a context value.

  6. def field[A](contextFromName: (String) => Option[A]): JsonContextMatcher[A]

    Context matcher that matches events within certain JSON object fields, extracting some context value based on the name of the matched field.

    Context matcher that matches events within certain JSON object fields, extracting some context value based on the name of the matched field. Specifically this matcher looks for an ObjectStart followed by a FieldStart(name), then plugs the name into contextFromName. If contextFromName returns a Some, the match succeeds and the value inside the Some is returned.

    contextFromName

    a function that extracts a context value based on the name of the JSON object field

  7. implicit def field(name: String): JsonContextMatcher[Unit]

    Context matcher that matches events within a given JSON object field.

    Context matcher that matches events within a given JSON object field. Specifically this matcher looks for an ObjectStart followed by a FieldStart(name) in the JSON context stack.

    When calling Splitter.json, the fact that this method is implicit allows you to use the underlying field name directly instead of directly calling this method, e.g. Splitter.json("address" \ "street")

    name

    the name of the field

  8. def fieldWhere(f: (String) => Boolean): JsonContextMatcher[String]

    Context matcher that matches events within JSON object fields whose names case the given f predicate to return true.

    Context matcher that matches events within JSON object fields whose names case the given f predicate to return true. Specifically this matcher looks for an ObjectStart followed by a FieldStart(name) where f(name) is true. If the match succeeds, the name of the field is also extracted as a context value.

    f

    A predicate applied to field names to determine whether the current JSON context stack matches

  9. def index[A](contextFromIndex: (Int) => Option[A]): JsonContextMatcher[A]

    Context matcher that matches events within a JSON array, at certain indexes.

    Context matcher that matches events within a JSON array, at certain indexes. Specifically this matcher looks for an ArrayStart followed by an IndexStart(i) in the JSON context stack, where the i from the IndexStart is passed to contextFromIndex. If contextFromIndex(i) returns a Some, the match succeeds and the value inside the Some is returned as a context value.

  10. def index(i: Int): JsonContextMatcher[Unit]

    Context matcher that matches events within a JSON array at a specific index.

    Context matcher that matches events within a JSON array at a specific index. Specifically this matcher looks for an ArrayStart followed by an IndexStart(i) in the JSON context stack.

  11. def indexWhere(f: (Int) => Boolean): JsonContextMatcher[Int]

    Context matcher that matches events within a JSON array, at certain indexes.

    Context matcher that matches events within a JSON array, at certain indexes. Specifically this matcher looks for an ArrayStart followed by an IndexStart(i) in the JSON context stack, where f(i) is true. If the match succeeds, the i index will be captured as a context value.

  12. implicit val jsonParserForPrimitiveBoolean: JsonParser[Boolean]

    Implicit version of JsonParser.forBoolean

  13. implicit val jsonParserForPrimitiveDouble: JsonParser[Double]

    Implicit version of JsonParser.forDouble

  14. implicit val jsonParserForPrimitiveFloat: JsonParser[Float]

    Implicit version of JsonParser.forFloat

  15. implicit val jsonParserForPrimitiveInt: JsonParser[Int]

    Implicit version of JsonParser.forInt

  16. implicit val jsonParserForPrimitiveLong: JsonParser[Long]

    Implicit version of JsonParser.forLong

  17. implicit val jsonParserForPrimitiveNull: JsonParser[None.type]

    Implicit version of JsonParser.forNull

  18. implicit val jsonParserForPrimitiveString: JsonParser[String]

    Implicit version of JsonParser.forString

  19. object Fs2DataSource

    Provides helpers for creating FS2 streams of io.dylemma.spac.json.JsonEvent, using fs2-data-json as the underlying event provider.

    Provides helpers for creating FS2 streams of io.dylemma.spac.json.JsonEvent, using fs2-data-json as the underlying event provider.

    This helper is set up to mirror the layout of the corresponding Fs2DataSource helper in the fs2-data-xml support module, but since there's less customization involved in setting up a stream of fs2-data-json Token objects, there's a lot less going on in this object. The main functionality provided here is the convert pipe, and some small helpers to automatically call it.

    For example:

    val charStream: Stream[IO, Char] = ???
    
    // this...
    val jsonStream1: Stream[IO, JsonEvent] = {
      import fs2.data.json._
      charStream
        .through(tokens)
        .through(Fs2DataSource.convertEvents)
    }
    
    // could be replaced with
    val jsonStream2: Stream[IO, JsonEvent] = {
      Fs2DataSource[IO](charStream)
    }

    Essentially this helper just provides a convenient apply method that accepts String, Stream[F, Char], Stream[F, String], or Stream[F, fs2.data.json.Token] to return a Stream[F, JsonEvent] which a JsonParser could then interact with.

  20. object JacksonSource

    Provides helpers for creating fs2.Stream and Iterator instances of JsonEvent from various underlying event sources, using the Jackson library as the underlying event parser.

    Provides helpers for creating fs2.Stream and Iterator instances of JsonEvent from various underlying event sources, using the Jackson library as the underlying event parser.

    The helpers in this object operate in terms of the IntoJacksonJsonParser typeclass, which defines logic for using a source value to construct a Jackson JsonParser in a Resource. From there, the helpers take care of converting the jackson parser/event model to the spac JsonEvent model.

    For example:

    val file = new File("./stuff.json")
    val jsonStream: Stream[IO, JsonEvent] = JacksonSource[IO](file)
  21. object JsonEvent

Inherited from AnyRef

Inherited from Any

JSON-specific extensions for Parser and Splitter

JSON-specific Type and Value aliases

JSON Context Matcher Construction

JSON Event Representation

Backend Parser Support