Q/PhdFddlZddlmZddlmZddlmZdededefdZdS) N) DataFrame)MapType)Settingsconfigdfreturncd|jjD}|r5tjdd|d|j|}|S)a=Preprocess the Spark DataFrame by removing MapType columns. - Excludes columns of type MapType - Raises a warning if any MapType columns are removed Args: config: Report settings object df: The Spark DataFrame Returns: The preprocessed DataFrame without MapType columns. cPg|]#}t|jt|j$S) isinstancedataTypername).0fields k/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/ydata_profiling/model/spark/dataframe_spark.py z$spark_preprocess..s;Ju~w4W4W z>Spark DataFrame profiling does not handle MapTypes. Column(s) z, z will be ignored. To fix this, consider converting your MapType into a StructType or extracting key-value pairs using `explode()`. )schemafieldswarningswarnjoindrop)rrcolumns_to_removes rspark_preprocessr s} " 0)  tyyYjOkOk      RW' ( Ir)r pyspark.sqlrpyspark.sql.typesrydata_profiling.configrrr rrrsw!!!!!!%%%%%%++++++X9r