Timestamp values are mapped 8 bytes long on Postgresql

[brbza]

No description provided.

[pbrumm]

We use mainly 9.2 with some 9.3. Let me check compatibility, I may need to add a flag to the encoder if this doesn't work on all of them.

[pbrumm]

to help me test this.

could you go into a psql prompt
on a test db

and run the following

create table test (ts timestamp without time zone); 
insert into test values ('2014-12-02 16:01:22.437311');
copy test to '/tmp/timestamp_test.dat' with binary;

and then send me the timestamp_test.dat file so I can run add a spec to test your example.
also if you could send me the table structure for the table your having problems with that would be helpful.

\d <table>

are you using the same column type as above?

http://www.postgresql.org/docs/9.3/static/datatype-datetime.html

[pbrumm]

I just did the above on a 9.3 postgres and my export is equivalent. and it is exporting it as 8bit.

I also don't see a difference in the output between L!> and Q!>.
Q!> is also a newer 2.1 feature.

[brbza]

Hi Pete,

Sorry for the delay on this answer...

As described on Postgresql documentation, timestamp with or without time
zone is stored as 8 bytes. First time I did the test I was using a legacy
database where the time field was defined as timestamp WITH time zone, now
I'm using a database built by an active records migration where a timestamp
type is created in Postgres as timestamp WITHOUT time zone. In both
situations it works for me when I use the pack method with the "Q"
directive.

Lookin into the Array Class documentation for the pack method (
http://www.ruby-doc.org/core-2.1.5/Array.html#method-i-pack) it states that:

C | Integer | 8-bit unsigned (unsigned char)
S | Integer | 16-bit unsigned, native endian (uint16_t)
L | Integer | 32-bit unsigned, native endian (uint32_t)
Q | Integer | 64-bit unsigned, native endian (uint64_t)

Note that the Q directive is available for older versions of Ruby too, just
the Q! is supported for version 2.1 onwards.

So, if we use pack with "L!>", "L", "L>", "L!<" or any other L directive we
will have the result in 4 bytes (32 bits). On the other hand, when using
the directives "Q", "Q>", "Q!>" or any other q directive we will have the
result in 8 bytes (64 bits) as expected by Postgresql.

We can test it using irb and running part of the gem code:

2.1.2 :001 > field = Time.now
=> 2014-12-04 14:37:13 -0200
2.1.2 :019 > POSTGRES_EPOCH_TIME = (Time.utc(2000,1,1).to_f *
1_000_000).to_i
=> 946684800000000
2.1.2 :020 > buf = [(field.to_f * 1_000_000 -
POSTGRES_EPOCH_TIME).to_i].pack("L!>")
=> "WL\x87G"
2.1.2 :021 > buf.bytesize

• => 4 *
  2.1.2 :022 > buf = [(field.to_f * 1_000_000 -
  POSTGRES_EPOCH_TIME).to_i].pack("Q!>")
  => "\x00\x01\xACeWL\x87G"
  2.1.2 :023 > buf.bytesize
• => 8*

even using just "Q>" we get the same result, I don't know if we need the
"!" here...

2.1.2 :024 > buf = [(field.to_f * 1_000_000 -
POSTGRES_EPOCH_TIME).to_i].pack("Q>")
=> "\x00\x01\xACeWL\x87G"
2.1.2 :025 > buf.bytesize
=> 8

Maybe is something related with the version of Ruby, I'm using ruby
2.1.2p95 (2014-05-08 revision 45877) [i686-linux] here... or maybe our
columns types are really different, I've tested using timestamp with and
without time zone on postgresql.

If you still need the binary export from postgresql I can provide you, that
was basically what I did to find out that the export has 8 bytes for a
timestamp column and the pack("L!>") returns just 4 bytes.

Best Regards,

Carlos

On Tue, Dec 2, 2014 at 8:38 PM, Pete Brumm [email protected] wrote:

I just did the above on a 9.3 postgres and my export is equivalent. and it
is exporting it as 8bit.

I also don't see a difference in the output between L!> and Q!>.

Q!> is also a newer 2.1 feature.

—
Reply to this email directly or view it on GitHub
#8 (comment).