Packages

  • package root
    Definition Classes
    root
  • package io
    Definition Classes
    root
  • package dylemma
    Definition Classes
    io
  • package spac

    SPaC (short for "Streaming Parser Combinators") is a library for building stream consumers in a declarative style, specialized for tree-like data types like XML and JSON.

    SPaC (short for "Streaming Parser Combinators") is a library for building stream consumers in a declarative style, specialized for tree-like data types like XML and JSON.

    Many utilities for handling XML and JSON data involve parsing the entire "document" to some DOM model, then inspecting and transforming that model to extract information. The downside to these utilities is that when the document is very large, the DOM may not fit in memory. The workaround for this type of problem is to treat the document as a stream of "events", e.g. "StartElement" and "EndElement" for XML, or "StartObject" and "EndObject" for JSON. The downside to this workaround is that writing code to handle these streams can be complicated and error-prone, especially when the DOM is complicated.

    SPaC's goal is to drastically simplify the process of creating code to handle these streams.

    This package contains the "core" SPaC traits; Parser, Transformer, Splitter, and ContextMatcher.

    See the xml and json subpackages (provided by the xml-spac and json-spac libraries respectively) for specific utilities related to handling XML and JSON event streams.

    Definition Classes
    dylemma
  • package xml

    This package provides extensions to the core "spac" library which allow for the handling of XML data.

    This package provides extensions to the core "spac" library which allow for the handling of XML data.

    Rather than creating explicit classes that extend Parser, Transformer, and Splitter, this package provides type aliases and implicit extensions. For example, XmlParser[A] is just a type alias for Parser[XmlEvent, A], and XmlParser is just a call to Parser[XmlEvent].

    Three main Parser methods are added to Parser[XmlEvent] via the XmlParserApplyOps implicit class:

    • XmlParser.forText - for capturing raw text
    • XmlParser.attr - for capturing mandatory attributes from elements
    • XmlParser.attrOpt - for capturing optional attributes from elements

    One main Splitter constructor method is added to Splitter via the XmlSplitterApplyOps implicit class:

    • Splitter.xml - for creating splitters based on an inspection of an "element stack"

    Three main Splitter member methods are added to Splitter[XmlEvent, C] via the XmlSplitterOps implicit class:

    • .attr - alias for .joinBy(XmlParser.attr(...))
    • .attrOpt - alias for .joinBy(XmlParser.attrOpt(...))
    • .text - alias for .joinBy(XmlParser.forText)

    A DSL for creating xml-specific ContextMatchers is provided to make it more convenient to call Splitter.xml. For example:

    Splitter.xml("things" \ "thing").attr("foo").parseToList

    Can be used to capture a list of the "foo" attributes in the <thing> elements in

    <things>
   <thing foo="hello" />
   <thing foo="Goodbye">
      <extra>junk</extra>
   </thing>
</thing>
    Definition Classes
    spac
  • AsQName
  • Fs2DataQName
  • Fs2DataSource
  • JavaxQName
  • JavaxSource
  • XmlEvent
  • XmlParserApplyOps
  • XmlSpacException
  • XmlSplitterApplyOps
  • XmlSplitterOps
o

io.dylemma.spac.xml

Fs2DataSource

object Fs2DataSource

Provides helpers for creating FS2 streams of io.dylemma.spac.xml.XmlEvent, using fs2-data-xml as the underlying event provider.

Fs2-data is already pretty "plug-and-play", so at best this helper just removes a few steps/pipes that you would have to manually call.

For example:

val charStream: Stream[IO, Char] = ???

// this...
val xmlStream1: Stream[IO, XmlEvent] = {
  import fs2.data.xml._
  charStream
    .through(events)
    .through(namespaceResolver)
    .through(reverenceResolver())
    .through(normalize)
    .through(Fs2DataSource.convert)
}

// could be replaced with
val xmlStream2: Stream[IO, XmlEvent] = {
  Fs2DataSource.fromRawXmlStream(charStream)
}

Essentially this helper just provides a convenient apply method that accepts String, Stream[F, Char], Stream[F, String], or Stream[F, fs2.data.xml.XmlEvent] to return a Stream[F, XmlEvent] which an XmlParser could then interact with.

Source
Fs2DataSource.scala
Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. Fs2DataSource
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Type Members

  1. trait Cleanup extends AnyRef

    Represents some post-processing on a stream of fs2-data-xml XmlEvent, e.g.

    Represents some post-processing on a stream of fs2-data-xml XmlEvent, e.g. piping the stream through a resolver and/or normalizer.

    This is how the ToFs2XmlEventStream typeclass allows you to override the stream creation behavior for the instances that use the events pipe themselves.

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.CloneNotSupportedException]) @native()
  6. def convert[F[_]]: Pipe[F, fs2.data.xml.XmlEvent, XmlEvent]

    Pipe for converting fs2.data.xml.XmlEvent to io.dylemma.spac.xml.XmlEvent.

    Pipe for converting fs2.data.xml.XmlEvent to io.dylemma.spac.xml.XmlEvent. This will be used under the hood of the apply and syncIO helpers, but is being made available to allow for manual stream creation if desired. The stream of fs2-data XmlEvents should ideally already be passed through a referenceResolver and normalize pipe before this one.

  7. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  8. def equals(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef → Any
  9. def finalize(): Unit
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.Throwable])
  10. def fromRawXmlStream[F[_], A](rawXmlStream: Stream[F, A], cleanup: Cleanup = Cleanup)(implicit A: CharLikeChunks[F, A], F: MonadError[F, Throwable]): Stream[F, XmlEvent]

    Converts a stream of raw XML data (such as Strings, Characters, or Bytes) to a stream of XmlEvents.

    Converts a stream of raw XML data (such as Strings, Characters, or Bytes) to a stream of XmlEvents.

    Under the hood, this calls events[F, A] from fs2-data-xml, piped through the cleanup functions (i.e. namespaceResolver, referenceResolver, normalize), then converts the fs2-data based XML events to the SPaC model of XmlEvent

    F

    Stream effect type

    A

    The chunk type of the raw XML data, i.e. String, Char, or Byte

    rawXmlStream

    The raw XML data

    cleanup

    Cleanup function used on the fs2-data XmlEvents. By default this will pass the events through namespaceResolver through referenceResolver() through normalize.

    A

    Evidence that the A type can be treated as raw xml by fs2-data-xml

    F

    Evidence that errors can be thrown in the F effect context

    returns

    A stream of SPaC XmlEvents

  11. def fromString[F[_]](rawXml: String, cleanup: Cleanup = Cleanup)(implicit F: MonadError[F, Throwable]): Stream[F, XmlEvent]

    Converts a raw XML string (expected to contain a complete XML document) to a stream of XmlEvents.

    Converts a raw XML string (expected to contain a complete XML document) to a stream of XmlEvents.

    F

    Stream effect type

    rawXml

    The raw XML data

    cleanup

    Cleanup function used on the fs2-data XmlEvents. By default this will pass the events through namespaceResolver through referenceResolver() through normalize.

    F

    Evidence that errors can be thrown in the F effect context

    returns

    A stream of SPaC XmlEvents

  12. final def getClass(): Class[_ <: AnyRef]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  13. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  14. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  15. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  16. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  17. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  18. final def synchronized[T0](arg0: => T0): T0
    Definition Classes
    AnyRef
  19. def toString(): String
    Definition Classes
    AnyRef → Any
  20. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])
  21. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])
  22. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException]) @native()
  23. object Cleanup extends Cleanup

    Default cleanup function which passes the input stream through namespaceResolver (with default args), then a default referenceResolver, then normalize

  24. object NoCleanup extends Cleanup

    No-op cleanup function which returns the input stream unmodified

Inherited from AnyRef

Inherited from Any

Ungrouped