Re: 250 thousand rows with CLOB ??

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: 250 thousand rows with CLOB ??

Amit.Shah

Forgot to mention, I use the latest hsqldb version – 2.2.8

 

From: Shah, Amit
Sent: Friday, April 20, 2012 3:51 PM
To: '[hidden email]'
Subject: 250 thousand rows with CLOB ??

 

I have a table with over 250K rows. One of the column is “description” and has rows that contain over 65K characters. Will it be possible to use hsqlDB at all ?

 

I tried to create table with CLOB and insert records. But the .lobs file reaches 2G and at that point onwards, it starts throwing “negative seek offset” error. Can I store the string as a BLOB ? Probably will have to worry about encoding .. ?? Thoughts ?

 

Thanks,

Amit


------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
_______________________________________________
Hsqldb-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/hsqldb-user
Reply | Threaded
Open this post in threaded view
|

Re: 250 thousand rows with CLOB ??

Fred Toussi-2
This usage has been tested before with larger .lobs files. There might be a memory issue without changing some of the lob schema tables to CACHED (see the Guide).
 
I will run the tests with large amounts of CLOB and BLOB in the next few days. If any issue shows up, it will be fixed and a snapshot jar with the fix will be issued.
 
Fred
 
On Fri, Apr 20, 2012, at 19:59, [hidden email] wrote:

Forgot to mention, I use the latest hsqldb version – 2.2.8

 

From: Shah, Amit
Sent: Friday, April 20, 2012 3:51 PM
To: '[hidden email]'
Subject: 250 thousand rows with CLOB ??

 

I have a table with over 250K rows. One of the column is “description” and has rows that contain over 65K characters. Will it be possible to use hsqlDB at all ?

 

I tried to create table with CLOB and insert records. But the .lobs file reaches 2G and at that point onwards, it starts throwing “negative seek offset” error. Can I store the string as a BLOB ? Probably will have to worry about encoding .. ?? Thoughts ?

 

Thanks,

Amit

------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
_______________________________________________
Hsqldb-user mailing list
 

------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
_______________________________________________
Hsqldb-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/hsqldb-user
Reply | Threaded
Open this post in threaded view
|

Re: 250 thousand rows with CLOB ??

Amit.Shah

I tried changing it to cached. Still got the same thing. Will give it a shot again to double check.

 

What surprises me is, the I have a text file with insert scripts for the entire table and that is only around 60M. So what is going into the .lobs file that makes it hit over 2G at only half way of insertion ..

 

From: Fred Toussi [mailto:[hidden email]]
Sent: Friday, April 20, 2012 4:29 PM
To: User discussions about HyperSQL Database Engine.
Subject: Re: [Hsqldb-user] 250 thousand rows with CLOB ??

 

This usage has been tested before with larger .lobs files. There might be a memory issue without changing some of the lob schema tables to CACHED (see the Guide).

 

I will run the tests with large amounts of CLOB and BLOB in the next few days. If any issue shows up, it will be fixed and a snapshot jar with the fix will be issued.

 

Fred

 

On Fri, Apr 20, 2012, at 19:59, [hidden email] wrote:

Forgot to mention, I use the latest hsqldb version – 2.2.8

 

From: Shah, Amit
Sent: Friday, April 20, 2012 3:51 PM
To: '[hidden email]'
Subject: 250 thousand rows with CLOB ??

 

I have a table with over 250K rows. One of the column is “description” and has rows that contain over 65K characters. Will it be possible to use hsqlDB at all ?

 

I tried to create table with CLOB and insert records. But the .lobs file reaches 2G and at that point onwards, it starts throwing “negative seek offset” error. Can I store the string as a BLOB ? Probably will have to worry about encoding .. ?? Thoughts ?

 

Thanks,

Amit

------------------------------------------------------------------------------

For Developers, A Lot Can Happen In A Second.

Boundary is the first to Know...and Tell You.

Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!

_______________________________________________

Hsqldb-user mailing list

 


------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
_______________________________________________
Hsqldb-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/hsqldb-user
Reply | Threaded
Open this post in threaded view
|

Re: 250 thousand rows with CLOB ??

Fred Toussi-2
You can use a smaller scale for the lobs file to reduce wasted space. The default is 32K, but you could use 4K instead.
 
Fred
 
On Fri, Apr 20, 2012, at 20:31, [hidden email] wrote:

I tried changing it to cached. Still got the same thing. Will give it a shot again to double check.

 

What surprises me is, the I have a text file with insert scripts for the entire table and that is only around 60M. So what is going into the .lobs file that makes it hit over 2G at only half way of insertion ..

 

From: Fred Toussi [mailto:[hidden email]]
Sent: Friday, April 20, 2012 4:29 PM
To: User discussions about HyperSQL Database Engine.
Subject: Re: [Hsqldb-user] 250 thousand rows with CLOB ??

 

This usage has been tested before with larger .lobs files. There might be a memory issue without changing some of the lob schema tables to CACHED (see the Guide).

 

I will run the tests with large amounts of CLOB and BLOB in the next few days. If any issue shows up, it will be fixed and a snapshot jar with the fix will be issued.

 

Fred

 

On Fri, Apr 20, 2012, at 19:59, [hidden email] wrote:

Forgot to mention, I use the latest hsqldb version – 2.2.8

 

From: Shah, Amit
Sent: Friday, April 20, 2012 3:51 PM
To: '[hidden email]'
Subject: 250 thousand rows with CLOB ??

 

I have a table with over 250K rows. One of the column is “description” and has rows that contain over 65K characters. Will it be possible to use hsqlDB at all ?

 

I tried to create table with CLOB and insert records. But the .lobs file reaches 2G and at that point onwards, it starts throwing “negative seek offset” error. Can I store the string as a BLOB ? Probably will have to worry about encoding .. ?? Thoughts ?

 

Thanks,

Amit

------------------------------------------------------------------------------

For Developers, A Lot Can Happen In A Second.

Boundary is the first to Know...and Tell You.

Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!

_______________________________________________

Hsqldb-user mailing list

 

------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
_______________________________________________
Hsqldb-user mailing list
 

------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
_______________________________________________
Hsqldb-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/hsqldb-user
Reply | Threaded
Open this post in threaded view
|

Re: 250 thousand rows with CLOB ??

Fred Toussi-2
In reply to this post by Fred Toussi-2
Thanks for reporting.
 
Initial tests showed this issue with clobs. The issue was fixed. Several extensive tests with up to 15GB in lobs completed successfully.
 
The latest snapshot jar includes the fix.
 
http://www.hsqldb.org/repos/org/hsqldb/hsqldb/SNAPSHOT/
 
Fred
 
On Fri, Apr 20, 2012, at 21:28, Fred Toussi wrote:
This usage has been tested before with larger .lobs files. There might be a memory issue without changing some of the lob schema tables to CACHED (see the Guide).
 
I will run the tests with large amounts of CLOB and BLOB in the next few days. If any issue shows up, it will be fixed and a snapshot jar with the fix will be issued.
 
Fred
 
On Fri, Apr 20, 2012, at 19:59, [hidden email] wrote:

Forgot to mention, I use the latest hsqldb version – 2.2.8

 

From: Shah, Amit
Sent: Friday, April 20, 2012 3:51 PM
To: '[hidden email]'
Subject: 250 thousand rows with CLOB ??

 

I have a table with over 250K rows. One of the column is “description” and has rows that contain over 65K characters. Will it be possible to use hsqlDB at all ?

 

I tried to create table with CLOB and insert records. But the .lobs file reaches 2G and at that point onwards, it starts throwing “negative seek offset” error. Can I store the string as a BLOB ? Probably will have to worry about encoding .. ?? Thoughts ?

 

Thanks,

Amit

------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
_______________________________________________
Hsqldb-user mailing list
 
------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
_______________________________________________
Hsqldb-user mailing list
 

------------------------------------------------------------------------------
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2

_______________________________________________
Hsqldb-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/hsqldb-user