Perl can be used to process RDD elements using pipes. Although this allows using Perl libraries for tokenization/parsing/etc., it is only a limited Perl integration in Spark. Notably:

1. A driver program in Python (Scala,Java) still has to exist.
2. Perl programs can operate only on individual RDD elements, meaning that more complex operations (reduceByKey, union, join, sortBy, i.e., operations defining order of multiple elements or joining of multiple elements) can be implemented in the driver program only.

Still, this functionality is useful when libraries available only in Perl have to be used.

Here we show how data can be passed from Python to Perl and back using JSON format, which allows preserving data types – RDD elements can be strings, numbers and array (note that Perl has no native tuples).

Using JSON format, we can easily serialize and deserialize the data we want to pass from Python to Perl and back. JSON format is used because:

1. It allows serializing numbers, strings and arrays.
2. The serialized JSON string contains no newlines, which fits the line-oriented Spark piping.
3. Libraries for JSON serialization/deserialization are available in both languages.

Scala and Java can be used in similar way as Python to communicate with Perl scripts via pipes. Nevertheless, available JSON libraries