Sign in to PrePAN

JSON::Delimited Provide stream and codec agnostic delimited JSON routines

In Review


# Delimited JSON decoding example

use common::sense;
use Data::Dumper;

#1. Pick a delimiting method
use JSON::Delimited::JSONL;		#Newline delimited
#use JSON::Delimited::JSONSeq;		#JSON Sequence delimited
#use JSON::Delimited::PrefixedJSON;	#Length prefixed JSON

#2. Pick your favourite JSON codec;
#    Package does not require a particular codec
use JSON;

#3. Create a buffer to reading/writing to
my $buffer;	

#4. Create a Delimited JSON object (line delimited in this example)
#   Use a reference to the buffer
#   Optionally set the encoder/decoder, as a simple wrapper sub on topic variable
#   to allow easy decoding instead of just extraction
my $jsonl=JSON::Delimited::JSONL->new->init(
	encoder=>sub {encode_json $_},
	decoder=>sub {decode_json $_}

#5. Setup input  reading loop
#   Read data is appended to buffer
binmode DATA;
my $size=4096;
while( sysread DATA,$buffer,$size,length $buffer){

	#6. Decode all JSON available in buffer.
	#   Only full JSON objects are consumed. Partial JSON reamin and 
	#   will be decoded with subsequent reads
	my @decoded=$jsonl->decodeMessages;
	say Dumper @decoded;

{"key":1, "value":2}
{"key":2, "value":4}
{"key":3, "value":8}


Provides a handful of different Delimited JSON methods operating on buffers:

  • JSONL (Newline delimited JSON)
  • JSONSeq (JSON Sequence)
  • PrefixedJSON (JSON with length prefixed)

Stream and codec agnostic

Handles the buffer manipulation to extract one or more JSON strings. Does not perform the JSON decoding itself, rather it utilises the modules specified by the user.

  • Users choice of JSON encoder/decoder
  • Users choice of data stream reading and writing.

Raw JSON or decoded data API

Either the JSON strings or the decoded objects can be extracted

Use cases

Intended to be used outside of HTTP/RPC (which also delimits JSON) in such use places as:

  • File storage
  • Pipe and socket IPC
  • Instead of a streaming decoder/encoder

Proposed Delimiting API (object to JSON string)

  • delimitMessage
  • delimitMessages
  • delimitMessagesTo
  • encodeMessage
  • encodeMessages
  • encodeMessagesTo

Proposed Extraction/Decoding API (JSON string to object)

  • extractNextMessage
  • extractMessages
  • extractMessagesTo
  • decodeNextMessage
  • decodeMessages
  • decodeMessagesTo


Please sign up to post a review.