pandas datetime день недели

pandas.to_datetime¶

Convert argument to datetime.

Parameters arg int, float, str, datetime, list, tuple, 1-d array, Series, DataFrame/dict-like

The object to convert to a datetime.

If ‘raise’, then invalid parsing will raise an exception.

If ‘coerce’, then invalid parsing will be set as NaT.

If ‘ignore’, then invalid parsing will return the input.

dayfirst bool, default False

Specify a date parse order if arg is str or its list-likes. If True, parses dates with the day first, eg 10/11/12 is parsed as 2012-11-10. Warning: dayfirst=True is not strict, but will prefer to parse with day first (this is a known bug, based on dateutil behavior).

yearfirst bool, default False

Specify a date parse order if arg is str or its list-likes.

If True parses dates with the year first, eg 10/11/12 is parsed as 2010-11-12.

If both dayfirst and yearfirst are True, yearfirst is preceded (same as dateutil).

Warning: yearfirst=True is not strict, but will prefer to parse with year first (this is a known bug, based on dateutil behavior).

utc bool, default None

Return UTC DatetimeIndex if True (converting any tz-aware datetime.datetime objects as well).

format str, default None

The strftime to parse time, eg “%d/%m/%Y”, note that “%f” will parse all the way up to nanoseconds. See strftime documentation for more information on choices: https://docs.python.org/3/library/datetime.html#strftime-and-strptime-behavior.

exact bool, True by default

unit str, default ‘ns’

The unit of the arg (D,s,ms,us,ns) denote the unit, which is an integer or float number. This will be based off the origin. Example, with unit=’ms’ and origin=’unix’ (the default), this would calculate the number of milliseconds to the unix epoch start.

infer_datetime_format bool, default False

If True and no format is given, attempt to infer the format of the datetime strings based on the first non-NaN element, and if it can be inferred, switch to a faster method of parsing them. In some cases this can increase the parsing speed by

origin scalar, default ‘unix’

Define the reference date. The numeric values would be parsed as number of units (defined by unit ) since this reference date.

If ‘unix’ (or POSIX) time; origin is set to 1970-01-01.

If ‘julian’, unit must be ‘D’, and origin is set to beginning of Julian Calendar. Julian day number 0 is assigned to the day starting at noon on January 1, 4713 BC.

If Timestamp convertible, origin is set to Timestamp identified by origin.

cache bool, default True

If True, use a cache of unique, converted dates to apply the datetime conversion. May produce significant speed-up when parsing duplicate date strings, especially ones with timezone offsets. The cache is only used when there are at least 50 values. The presence of out-of-bounds values will render the cache unusable and may slow down parsing.

If parsing succeeded. Return type depends on input:

Series: Series of datetime64 dtype

In case when it is not possible to return designated types (e.g. when any element of input is before Timestamp.min or after Timestamp.max) return will have datetime.datetime type (or corresponding array/Series).

Cast argument to a specified dtype.

Convert argument to timedelta.

Assembling a datetime from multiple columns of a DataFrame. The keys can be common abbreviations like [‘year’, ‘month’, ‘day’, ‘minute’, ‘second’, ‘ms’, ‘us’, ‘ns’]) or plurals of the same

If a date does not meet the timestamp limitations, passing errors=’ignore’ will return the original input instead of raising any exception.

Passing errors=’coerce’ will force an out-of-bounds date to NaT, in addition to forcing non-dates (or non-parseable dates) to NaT.

Passing infer_datetime_format=True can often-times speedup a parsing if its not an ISO8601 format exactly, but in a regular format.

Источник

Working with Pandas datetime

In this post we will explore the Pandas datetime methods which can be used instantaneously to work with datetime in Pandas.

I am sharing the table of content in case you are just interested to see a specific topic then this would help you to jump directly over there

Import time-series data

This is the monthly electrical consumption data in csv which we will import in a dataframe for this tutorial and this data can be downloaded using this link

parse_dates attributes in read_csv() function

We are using **parse_date** attribute to parse and convert the date columns in the csv files to numpy datetime64 type

Pandas to_datetime

Alternatively, you can use to_datetime to convert any column to datetime

Extract Month and Year from datetime using datetime accessor

We will create 3 new columns here for Year, Month and day after extracting it from the Date column

Time Series- Aggregation

Resample to find sum on the date index date

resample() is a method in pandas that can be used to summarize data by date or time

Before re-sampling ensure that the index is set to datetime index i.e. DATE column here

Let’s find the Yearly sum of Electricity Consumption

Resample to find mean on the date index date

Lets find the Electricity consumption mean for each year

Datetime index and slice

Just ensure that the datetime column is set as index for the dataframe. I am using set_index() function to set that before index and slice

Filter using the date

Get all the rows for year 1987

Filter all rows between two dates i.e. 1989-JAN and 1995-Apr here

Get all rows between JAN-1989 and APR-1995

Date Offset

Its a kind of date increment used for a date range.

As per the documentation: Each offset specify a set of dates that conform to the DateOffset.

For example, Bday defines this set to be the set of dates that are weekdays (M-F). To test if a date is in the set of a DateOffset dateOffset we can use the onOffset method: dateOffset.onOffset(date).

If a date is not on a valid date, the rollback and rollforward methods can be used to roll the date to the nearest valid date before/after the date

DateOffsets can be created to move dates forward a given number of valid dates.

For example, Bday(2) can be added to a date to move it two business days forward. If the date does not start on a valid date, first it is moved to a valid date

Add a day to DATE Column

Here we are adding a day(timedelta of 1 day) to the Date column in dataframe and creating a new column called as next_day

Add a Business day to DATE Column

Here we are adding a Business day using Bday param, it will add a day between Mon-Fri.

if a date is Sat then adding a Bday will return the next Monday i.e. a Business day instead of a Saturday

Add 2 business days to DATE Column

Adding two days to the current DATE column using days parameter and create a new column day_after

Add next month date

Adding a month to the DATE column using months parameter

For the complete list of parameters check this link

Using date_range to create datetime index

it is Immutable numpy ndarray of datetime64 data.

We will see how to create datetime index and eventually create a dataframe using these datetime index arrays

Datetime index with Hourly frequency

It gives the array of date and time starting from ‘2018-01-01’ with a Hourly frequency and period=3 means total elements of 3

DatetimeIndex([‘2018-01-01 00:00:00’, ‘2018-01-01 01:00:00’, ‘2018-01-01 02:00:00’], dtype=’datetime64[ns]’, freq=’H’)

Monthly Frequency

Now change the frequency to Monthly and create array of total 10 dates

Weekly Frequency with start and end

Change the frequency to Weekly and create dates between two dates using start and end dates

Источник

Как использовать модуль datetime в Python

Datetime — важный элемент любой программы, написанной на Python. Этот модуль позволяет управлять датами и временем, представляя их в таком виде, в котором пользователи смогут их понимать.

datetime включает различные компоненты. Так, он состоит из объектов следующих типов:

Как получить текущие дату и время?

Получить текущую дату в Python

Класс date можно использовать для получения или изменения объектов даты. Например, для получения текущей с учетом настроек подойдет следующее:

Текущая дата — 2020-11-14 в формате год-месяц-день соответственно.

Получить текущее время

Для получения текущего локального времени сперва нужно получить текущие дату и время, а затем достать из этого объекта только время с помощью метода time() :

Компоненты datetime в Python

В этом руководстве речь пойдет о следующих элементах:

Как создавать объекты даты и времени

В этом примере создается объект времени представленный следующим образом (8, 48, 45).

Для создания объекта даты нужно передать дату с использованием следующего синтаксиса:

Вернет вот такой результат:

Timedelta

Все аргументы опциональные и их значения по умолчанию равно 0. Они могут быть целыми или числами с плавающей точкой, как положительными, так и отрицательными. Благодаря этому можно выполнять математические операции, такие как сложение, вычитание и умножение.

Как вычислить разницу для двух дат

Посмотрим на несколько примеров вычисления разницы во времени. Предположим, есть два объекта datetime :

Для получения разницы нужно лишь вычесть значение одного объекта из второго:

Таким образом между 2 и 30 октября 2020 года 28 дней.

Как вычислить разницу двух объектов datetime.time

Такой код вернет следующую ошибку:

Как получать прошлые и будущие даты с помощью timedelta

Поскольку timedelta — это длительность, то для получения прошлой или будущей даты нужно добавить объект timedelta к существующему или вычесть из него же. Вот пример нескольких уравнений, где n — это целое число, представляющее количество дней:

Если нужно, например, получить дату за прошлые две недели, то достаточно вычесть 14 дней из текущей даты:

Предположим, вы задумали практиковать определенный навык в течение 21 дня. Для получения будущей даты нужно добавить 21 день к текущей дате:

Другие арифметические операции с timedelta

Значения даты и времени могут сравниваться для определения того, какая из них была раньше или позже. Например:

Часовые пояса

Пока что мы работали с datetime без учета часовых поясов и летнего времени. Но прежде чем переходить к следующим пунктам, нужно разобраться с разницей в абсолютных (naive) и относительных (aware) датах.

Абсолютные даты не содержат информацию, которая бы могла определить часовой пояс или летнее время. Однако с такими намного проще работать.

Относительные же содержат достаточно информации для определения часового пояса или отслеживания изменений из-за летнего времени.

Разница между DST, GMT и UTC

Как работать с часовыми поясами

Рассмотрим, как создать простой относительный объект datetime :

Предположим, нужно получить текущее время для Найроби. Для этого нужно использовать конкретный часовой пояс. Для начала можно с помощью pytz получить все существующие часовые пояса.

Вот некоторые из них:

Для получения времени в Найроби:

А вот так можно получить время Берлина:

Здесь можно увидеть разницу в часовых поясах разных городов, хотя сама дата одна и та же.

Конвертация часовых поясов

При конвертации часовых поясов в первую очередь нужно помнить о том, что все атрибуты представлены в UTC. Допустим, нужно конвертировать это значение в America/New_York :

Другие практические примеры

Всегда храните даты в UTC. Вот примеры:

Как конвертировать строки в datetime

strptime() в Python — это метод из модуля datetime. Вот его синтаксис:

Аргументы формата необязательные и являются строками. Предположим, нужно извлечь текущие дату и время:

Источник

Pandas: How to create a datetime object from Week and Year?

I have a dataframe that provides two integer columns with the Year and Week of the year:

I need to create a datetime-object from these two numbers.

I tried this, but it throws an error:

Then I tried this, it works but gives the wrong result, that is it ignores the week completely:

I’m using Python 3, if that is relevant in any way.

Starting with Python 3.8 the problem is easily solved with a newly introduced method on datetime.date objects: https://docs.python.org/3/library/datetime.html#datetime.date.fromisocalendar

4 Answers 4

Initially I have timestamps in s

It’s much easier to parse it from UNIX epoch timestamp:

Timing for 10M rows DF:

Conclusion: I think 156 milliseconds for converting 10.000.000 rows is not that slow

Like @Gianmario Spacagna mentioned for datetimes higher like 2018 use %V with %G :

There is something fishy going on with weeks starting from 2019. The ISO-8601 standard assigns the 31st December 2018 to the week 1 of year 2019. The other approaches based on:

will give shifted results starting from 2019.

In order to be compliant with the ISO-8601 standard you would have to do the following:

The week 53 of 2018 is ignored and mapped to the week 1 of 2019.

If you want to follow ISO Week Date

Weeks start with Monday. Each week’s year is the Gregorian year in which the Thursday falls. The first week of the year, hence, always contains 4 January. ISO week year numbering therefore slightly deviates from the Gregorian for some days close to 1 January.

The following sample code, generates a sequence of 60 Dates, starting from 18Dec2016 Sun and adds the appropriate columns.

Источник

Working with datetime in Pandas DataFrame

Some Pandas tricks to help you get started with data analysis

Aug 28, 2020 · 8 min read

Datetime is a common data type in data science projects. Often, you’ll work with it and run into problems. I found Pandas is an amazing library that contains extensive capabilities and features for working with date and time.

In this article, we will cover the following common datetime problems and should help you get started with data analysis.

Please check out my Github repo for the source code.

1. Convert strings to datetime

Pandas has a built-in function called to_datetime() that can be used to convert strings to datetime. Let’s take a look at some examples

With default arguments

Pandas to _ datetime() is able to parse any valid date string to datetime without any additional arguments. For example:

Day first format

By default, to_datetime() will parse string with month first ( MM/DD, MM DD, or MM-DD) format, and this arrangement is relatively unique in the United State.

Custome format

Speed up parsing with infer_datetime_format

Passing infer_datetime_format=True can often speed up a parsing if its not an ISO8601 format exactly but in a regular format. According to [1], in some cases, this can increase the parsing speed by 5–10x.

Handle parsing error

You will end up with a TypeError if the date string does not meet the timestamp format.

In addition, if you would like to parse date columns when reading data from a CSV file, please check out the following article

4 tricks you should know to parse date columns with Pandas read_csv()

Some of the most helpful Pandas tricks

2. Assemble a datetime from multiple columns

to_datetime() can be used to assemble a datetime from multiple columns as well. The keys (columns label) can be common abbreviations like [‘year’, ‘month’, ‘day’, ‘minute’, ‘second’, ‘ms’, ‘us’, ‘ns’]) or plurals of the same.

3. Get year, month, and day

First, let’s create a dummy DateFrame and parse DoB to datetime.

And to get year, month, and day

4. Get the week of year, the day of week and leap year

Note that Pandas dt.dayofweek attribute returns the day of the week and it is assumed the week starts on Monday, which is denoted by 0 and ends on Sunday which is denoted by 6. To replace the number with full name, we can create a mapping and pass it to map() :

5. Get the age from the date of birth

The simplest solution to get age is by subtracting year:

However, this is not accurate as people might haven’t had their birthday this year. A more accurate solution would be to consider the birthday

6. Improve performance by setting date column as the index

A common solution to select data by date is using a boolean maks. For example

To set the date column as the index

7. Select data with a specific year and perform aggregation

Let’s say we would like to select all data in the year 2018

And to perform aggregation on the selection for example:

Get the total num in 2018

Get the total num for each city in 2018

8. Select data with a specific month and a specific day of the month

To select data with a specific month, for example, May 2018

Similarly, to select data with a specific day of the month, for example, 1st May 2018

9 Select data between two dates

To select data between two dates, you can use df.loc[start_date:end_date] For example:

Select data between 2016 and 2018

Select data between 10 and 11 o’clock on the 2nd May 2018

Select data between 10:30 and 10:45 on the 2nd May 2018

10 Handle missing values

We often need to compute window statistics such as a rolling mean or a rolling sum.

Let’s compute the rolling sum over a 3 window period and then have a look at the top 5 rows.

We can see that it only starts having valid values when there are 3 periods over which to look back. One solution to handle this is by backfilling of data.

For more details about backfilling, please check out the following article

Источник