Spark streaming checkpoint implementation and how it bit me (medium.com)

What I’ve learned about spark streaming checkpointing, how it’s implemented, and how the implementation details of offsets might lead to surprising behavior.