Towards a GraphQL Proxy for Apache Kafka
Date
2020
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Tartu Ülikool
Abstract
Apache Kafka is a open-source stream-processing framework which is quickly becoming
an industry standard for event-driven applications. Kafka is written in Scala and, thus,
general purpose JVM (java virtual machine) programming languages can be used to write
native applications that interact with a Kafka cluster using the Kafka protocol. However,
the need to generalize the interaction to other languages, e.g. python, has pushed Kafka
developers to abstract the Kafka protocol using the REST API. The REST proxy is a
component of the Kafka suite that enables the use of a RESTful (representational state
transfer) API (application programming interface) to communicate with a Kafka cluster,
and thus allowing to control Kafka using HTTP.
GraphQL is a data query and manipulation language for APIs, and a runtime for
fulfilling queries with existing data [1]. Recently GraphQL has been gaining traction.
GraphQL enables developers to define which data will be returned by a request, moreover
GraphQL allows to interact with this data type through a full-fledged query language.
In this thesis we propose the GraphQL proxy, which enables communication with a
Kafka cluster using GraphQL.
In estonian: Apache Kafka on avatud lähtekoodiga voogtöötluse raamistik, millest on saamas tööstusstandard
südmustepõhiste rakenduste arendamisel. Kafka on kirjutatud Scalas ning
seega saab kasutada üldotstarbelisi JVM (java virtuaalmasin) programmeerimiskeeli, et
kirjutada rakendusi, mis töötavad Kafka protokolliga. Sellegipoolest, on vajadus Kafka
laiendamiseks teistesse keeltesse, näiteks pyhon, pannud Kafka arendajad välja töötama
Kafka jaoks REST (representational state transfer) API (application programming interface).
REST proxy on Kafka komponent, mis lubab kasutada RESTful APIt, et suhelda
Kafka klastriga ning seega lubab kasutada Kafkat üle HTTP.
Hiljuti on populaarsust kogunud veebiliides GraphQL. GraphQL on avatud lähtekoodiga
andmepäringu keel, mis lubab arendajatel rangelt defineerida päringule vastu saadetavad
andmed. Kuna Avro skeemid on väga sarnased GraphQLi skeemidele, siis võib
GraphQL olla parem viist Kafka klastriga suhtlemiseks kui REST.
Description
Keywords
Kafka, GraphQL, web, streams, Avro, REST, veeb, vood, striimimine