Towards a GraphQL Proxy for Apache Kafka

Date

2020

Journal Title

Journal ISSN

Volume Title

Publisher

Tartu Ülikool

Abstract

Apache Kafka is a open-source stream-processing framework which is quickly becoming an industry standard for event-driven applications. Kafka is written in Scala and, thus, general purpose JVM (java virtual machine) programming languages can be used to write native applications that interact with a Kafka cluster using the Kafka protocol. However, the need to generalize the interaction to other languages, e.g. python, has pushed Kafka developers to abstract the Kafka protocol using the REST API. The REST proxy is a component of the Kafka suite that enables the use of a RESTful (representational state transfer) API (application programming interface) to communicate with a Kafka cluster, and thus allowing to control Kafka using HTTP. GraphQL is a data query and manipulation language for APIs, and a runtime for fulfilling queries with existing data [1]. Recently GraphQL has been gaining traction. GraphQL enables developers to define which data will be returned by a request, moreover GraphQL allows to interact with this data type through a full-fledged query language. In this thesis we propose the GraphQL proxy, which enables communication with a Kafka cluster using GraphQL. In estonian: Apache Kafka on avatud lähtekoodiga voogtöötluse raamistik, millest on saamas tööstusstandard südmustepõhiste rakenduste arendamisel. Kafka on kirjutatud Scalas ning seega saab kasutada üldotstarbelisi JVM (java virtuaalmasin) programmeerimiskeeli, et kirjutada rakendusi, mis töötavad Kafka protokolliga. Sellegipoolest, on vajadus Kafka laiendamiseks teistesse keeltesse, näiteks pyhon, pannud Kafka arendajad välja töötama Kafka jaoks REST (representational state transfer) API (application programming interface). REST proxy on Kafka komponent, mis lubab kasutada RESTful APIt, et suhelda Kafka klastriga ning seega lubab kasutada Kafkat üle HTTP. Hiljuti on populaarsust kogunud veebiliides GraphQL. GraphQL on avatud lähtekoodiga andmepäringu keel, mis lubab arendajatel rangelt defineerida päringule vastu saadetavad andmed. Kuna Avro skeemid on väga sarnased GraphQLi skeemidele, siis võib GraphQL olla parem viist Kafka klastriga suhtlemiseks kui REST.

Description

Keywords

Kafka, GraphQL, web, streams, Avro, REST, veeb, vood, striimimine

Citation