Press "Enter" to skip to content

Generating an Expression Variable for Joins with PySpark

Unmesha Sreeveni uses a variable to effect a join in PySpark:

 Lets see how to join 2 table with a parameterized on condition in PySpark

Eg: I have 2 dataframes A and B and I want to join them with id,inv_no,item and subitem

Click through to see how. It turns out to be pretty straightforward.