question

Upvotes
Accepted
hiroko.goto avatar image
1 ●0 ●1 ●2

File size in Python Pickle format

One of our clients would like to understand file size of TickHistory data to determine data capacity in their database.

They would like to know the size of data in Python pickle files through REST API.

Is it possible to guess Python pickle file size from csv file size in TickHistory GUI?

For example, some files are 7.4GB and 29.7GB respectively in csv format from TickHistory GUI (after unzipped from gunzip format).

In that case, roughly how much data capacity is expected in Python pickle format?

Regards,

Hiroko

tick-history-rest-api
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

1 Answer

· Write an Answer
Upvote
Accepted
zoya faberov avatar image
32.2k ●40 ●11 ●19

Hello @hiroko.goto ,

We do not have the statistics on pickle sizes that are specific to Tick History files.

I am looking at this relevant article, the author has done some relevant research, and from their findings pickle file looks to be slightly under half the size of CSV.

So gzip will be significantly more efficient, if they are looking to keep the data as blobs.

Hope that this information helps

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

hiroko.goto avatar image hiroko.goto
Hi Zoya,,


Thank you very much!

Yes, your information really helpful.


Best Regards,


Hiroko

Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.