The missing piece in the Data Engineer's toolset
This is done by Data Engineers who write code to build data infrastructure, to transform the data, to configure the pipelines, to create custom tools specific for their company's needs.
And yet the Notebooks used to write that code provide close to no assistance. A typo in a method name could results in hours of work wasted - waiting for that Spark job to finish only to crash on the last line. It doesn't have to be that way. At JetBrains, we have vast experience in intelligent coding assistance, and we're ready to bring this experience to the Spark world.