A monthly overview of things you need to know as an architect or aspiring architect. Unlock the full InfoQ experience by logging in! Stay updated with your favorite authors and topics, engage with ...

Another year passes. I was hoping to write more articles instead of just these end-of-the-year screeds, but I almost died in the spring semester, and it sucked up my time. Nevertheless, I will go ...

This project provides extensions to the Apache Spark project in Scala and Python: Diff: A diff transformation and application for Datasets that computes the differences between two datasets, i.e.

Graph processing serves as the essential building block for a diverse variety of real-world applications such as social network analytics, data mining, network routing, and scientific computing.

At the heart of Apache Spark is the concept of the Resilient Distributed Dataset (RDD), a programming abstraction that represents an immutable collection of objects that can be split across a ...