fc.dt.*.
Extraction Functions
year
Extract the year from a date column.The column to extract the year from.
A Column object with the year extracted.
Example
month
Extract the month from a date column.day
Extract the day from a date column.hour
Extract the hour from a timestamp column.This will return 0 for DateType columns.
minute
Extract the minute from a timestamp column.second
Extract the second from a timestamp column.millisecond
Extract the millisecond from a timestamp column.Conversion Functions
to_date
Transform a string into a DateType.The column to transform into a DateType.
The format of the date string. If not provided, the default format is “YYYY-MM-DD”. The accepted formats should follow the Spark datetime pattern.
A Column object with the DateType transformed.
Example
to_timestamp
Transform a string into a TimestampType.The column to transform into a TimestampType.
The format of the timestamp string. If not provided, the default format is ISO 8601 with milliseconds. The accepted formats should follow the Spark datetime pattern.
A Column object with the
TimestampType type, with a UTC timezone. If the provided format contains a timezone specifier, the result timestamp value will be converted from the format timezone to UTC.Example
date_format
Formats a date/timestamp column to a given format.The column to format.
The format to format the column to. The accepted formats should follow the Spark datetime pattern.
A Column object with the date/timestamp column formatted into a string.
Example
Current Time Functions
now
Get the current date and time.A Column object with the current date and time. The type of the column is TimestampType.
current_timestamp
Get the current date and time.A Column object with the current date and time. The type of the column is TimestampType in UTC timezone.
current_date
Get the current date.A Column object with the current date. The type of the column is DateType.
Arithmetic Functions
date_add
Adds the number of days to the date/timestamp column.The column to add the days to.
The number of days to add to the date/timestamp column. If the days is negative, the days will be subtracted.
A Column object with the date/timestamp column with the days added.
Example
date_sub
Subtracts the number of days from the date/timestamp column.The column to subtract the days from.
The amount of days to subtract. If the days is negative, the days will be added.
A Column object with the date/timestamp column with the days subtracted.
timestamp_add
Adds the quantity of the given unit to the timestamp column.The column to add the quantity to.
The quantity to add. If the quantity is negative, the quantity will be subtracted.
The unit of the quantity. Supported units:
"year", "month", "day", "hour", "minute", "second", "millisecond".A Column object with the timestamp column with the quantity added.
Example
datediff
Calculates the number of days between two date/timestamp columns.To date column to work on.
From date column to work on.
A Column object with the difference in days between the two date/timestamp columns.
Example
timestamp_diff
Calculates the difference between two timestamp columns.The first column to calculate the difference from.
The second column to calculate the difference from.
The unit of the difference. Supported units:
"year", "month", "day", "hour", "minute", "second", "millisecond".A Column object with the difference in the given unit between the two timestamp columns.
Truncation Functions
date_trunc
Truncate a date to a given unit.The column to truncate.
The unit to truncate to. Supported units:
"year", "month", "day", "hour", "minute", "second", "millisecond".A Column object with the date truncated.
Example
Timezone Functions
to_utc_timestamp
Accepts a Column with TimestampType (UTC), interprets each value as wall-clock time in the specified timezone, and converts it to a timestamp in UTC.The column containing the timestamp. Will be treated as timezone-agnostic.
A timezone that the input should be converted to.
A Column object with timestamp expressed in UTC.
In fenic, the TimestampType data type is always in UTC. This function treats the input timestamp as timezone-agnostic, sets it to the requested timezone (without changing the timestamp), then converts the timestamp to UTC.See Spark documentation for more details.
from_utc_timestamp
Accepts a Column with TimestampType (UTC). For each row, converts the timestamp value to the provided timezone, then renders that timestamp as UTC without changing the timestamp value.The column containing the timestamp.
A timezone that the input will be converted from.
A Column object with timestamp expressed in UTC.
See Spark documentation for more details.
