Home/ Questions/Q 1164
Next
In Process
dishach
  • 1
Explainer
Asked: 2023-07-01T09:22:19+00:00 In:

Based on several conditions, how can we replace one column value with another one in Pyspark?

  • 1

I want to replace the column values if the values in column one are NULL and the values in the other two columns are the same. After that the column four value in the data frame has to be placed at column one.

programspysparkpython
Share
Leave an answer

Leave an answer

Browse

1 Answer

  1. Professional

    For NULL condition we use, isnull() and .otherwise( df.col1 )

    For example:

    df = df.withColumn("col1", when( isnull(df.col1) & (df.col2 == df.col3), df.col4)).otherwise( df.col1 )

     

    • 0