Blog | luminousmen

Back

The ultimate Python style guidelines

Coding guidelines help engineering teams to write consistent code which is easy to read and understand for all team members. Python has an excellent style guide called PEP8. It covers most of the situations you will step into while writing Python. I like PEP8, I believe there has been much...

Introduction to Pyspark join types

This article is written to visualize various types of joins, a cheat sheet so that all join types are listed in the same place with examples and without stupid circles. Aaaah, circles! I've tired of these explanations of joins with intersections of sets and circles. It seems to be both clear and...

The 5-minute guide to using bucketing in Pyspark

There are many different tools in the world, each of which solves a range of problems. Many of them are judged by how well and correct they solve this or that problem, but there are tools that you just like, you want to use them. They are properly designed and fit well in your hand, you do not...

Spark tips. Don't collect data on driver

There are many different tools in the world, each of which solves a range of problems. Many of them are judged by how well and correct they solve this or that problem, but there are tools that you just like, you want to use them. They are properly designed and fit well in your hand, you do not...

How to not leap in time using Python

If you want to display the time to a user of your application, you query the time of day. However, if your application needs to measure elapsed time, you need a timer that will give the right answer even if the user changes the time on the system clock. The system clock which tells the time of...

Spark tips. DataFrame API

There are many different tools in the world, each of which solves a range of problems. Many of them are judged by how well and correct they solve this or that problem, but there are tools that you just like, you want to use them. They are properly designed and fit well in your hand, you do not...

Spark. Anatomy of Spark application

Apache Spark is considered as a powerful complement to Hadoop, big data’s original technology. Spark is a more accessible, powerful and capable big data tool for tackling various big data challenges. It has become mainstream and the most in-demand big data framework across all major industries....

Spark core concepts explained

Apache Spark is considered as a powerful complement to Hadoop, big data’s original technology. Spark is a more accessible, powerful and capable big data tool for tackling various big data challenges. It has become mainstream and the most in-demand big data framework across all major industries....

Best blogs/podcasts to follow for Python developers

I want to share blogs and podcasts I occasionally read/hear stuff from. Python subreddit First things first. This is a good feed for any Python related news. Dev.to This is a great source of information for beginners and I really love the community around this resource. Ned Batchelder blog This...

Asynchronous programming. Python3.5+

This is a practical post of the series of asynchronous programming. Whole series: Asynchronous programming. Blocking I/O and non-blocking I/O Asynchronous programming. Cooperative multitasking Asynchronous programming. Await the Future Asynchronous programming. Python3.5+ In this post, we will...