Icon View Thread

The following is the text of the current message along with any replies.
Messages 11 to 20 of 28 total
Thread A Possible Upcoming Breaking Change Concerning In-Memory Databases
Sun, Jan 13 2013 12:04 PMPermanent Link

Fernando Dias

Team Elevate Team Elevate

Raul,

<< There are scenarios where i believe it to be useful - for example mixing a remote C/S session and local session >>

I don't think it's going to work like that, as the local sessions are going to only see the local memory databases because they are created in the client computer memory. The memory databases are going to be visible to all the *server* sessions, not to all the client application sessions.

--
Fernando Dias
[Team Elevate]
Sun, Jan 13 2013 12:16 PMPermanent Link

Fernando Dias

Team Elevate Team Elevate

Tim,

The only advantage of such a change would be to have the possibility to share memory databases, but it would also mean we can't count on the isolation that we have now, and the process isolation is sometimes useful. The ideal would be to have both, and let the user choose the isolation level when the database is created, but I'm not sure if it's worth the extra code in the engine... In my opinion it doesn't, but that's only my opinion of course.

I can live with that tiny memory leak and after all we know how to avoid it.
Also, you will still have to decide when to drop the memory database catalogs, and if the isolation level is not the client process then when are you going to drop the catalogs ?

--
Fernando Dias
[Team Elevate]
Sun, Jan 13 2013 1:42 PMPermanent Link

Barry

>Strangely over the years where I've used in-memory tables I haven't used SQL on them. That's why when I built my own string table system I've built in simple sorting and filtering but that's all. A number of them do get populated using SQL though.<

Roy,

You need to think outside the "table box" (TEDBTable) and go with SQL.  For example, you can do a disk table to memory table, table joins. Just explicitly specify the database for the memory table using SQL as in

... From DiskTable DT left join MemDatabase.MemTable MT on DT.Cust_Id=MT.Cust_Id ...

This should speed up joins with often used tables.

Barry
Mon, Jan 14 2013 5:26 AMPermanent Link

Roy Lambert

NLH Associates

Team Elevate Team Elevate

Barry


>You need to think outside the "table box" (TEDBTable) and go with SQL. For example, you can do a disk table to memory table, table joins. Just explicitly specify the database for the memory table using SQL as in
>
> ... From DiskTable DT left join MemDatabase.MemTable MT on DT.Cust_Id=MT.Cust_Id ...
>
>This should speed up joins with often used tables.

Well it might if I needed to do something like that Smiley

I would counter your suggestion with

You need to think outside the "SQL box" - you can do lots of useful things in a fast manner using tables that aren't possible with an SQL result set (try editing an insensitive result set for starters).

Never forget SQL means Structured QUERY Language <vbg>

Roy
Mon, Jan 14 2013 8:40 AMPermanent Link

Tim Young [Elevate Software]

Elevate Software, Inc.

Avatar

Email timyoung@elevatesoft.com

Roy,

<< The way I read Tim's post is that the idea of shareable in-memory
databases would ONLY be implemented for c/s. >>

Correct.

Tim Young
Elevate Software
www.elevatesoft.com
Mon, Jan 14 2013 8:41 AMPermanent Link

Tim Young [Elevate Software]

Elevate Software, Inc.

Avatar

Email timyoung@elevatesoft.com

Fernando,

<< I don't think it's going to work like that, as the local sessions are
going to only see the local memory databases because they are created in the
client computer memory. The memory databases are going to be visible to all
the *server* sessions, not to all the client application sessions. >>

Correct.  There would be a difference between local applications and C/S
applications where, right now, none exists (apart from the small leak).

Tim Young
Elevate Software
www.elevatesoft.com
Mon, Jan 14 2013 9:01 AMPermanent Link

Tim Young [Elevate Software]

Elevate Software, Inc.

Avatar

Email timyoung@elevatesoft.com

Barry,

<< I had (incorrectly) assumed if memory tables were created with a Create
statement that did not use "Temporary" they were shareable. And if memory
tables were created with the "Temporary" keyword then they would be created
on the client's machine like any temporary table. >>

Just to clarify, temporary tables are never created on the local machine
with remote sessions.  They are created on the server, but isolated by
session.

<< This would be consistent with how EDB handles disk based tables. So if a
developer wants isolated memory tables, he would use "Create Temporary
MemTable ..." and if he wanted a shareable memory table he would use "Create
MemTable ...". >>

Correct, and temporary tables created in an in-memory database are also
stored in-memory.

<< MySQL can share Memory tables on the server, I can't see why EDB can't
either. I thought this was the whole purpose of using memory tables on a
server. Otherwise, as many have pointed out, you might as well be using EDB
temporary tables instead of memory tables because there would be little
difference between the two because they are non-shareable. >>

There is a small-subset of applications that do need per-process, in-memory
databases, but they can still just use local sessions for those since they
don't need need the in-memory tables to be on the EDB Server.

Thanks,

Tim Young
Elevate Software
www.elevatesoft.com
Mon, Jan 14 2013 9:02 AMPermanent Link

Tim Young [Elevate Software]

Elevate Software, Inc.

Avatar

Email timyoung@elevatesoft.com

Heiko,

<< Reason why I'm not using disk-based temporary tables is mainly the
belief, that memory is faster than disk, that "small" temporary stuff should
happen in RAM and not on disk (may it only be to prolong the lifetime of the
disk, matters more with SSDs), and that, when you restart the machine, all
temporary stuff that wasn't cleaned up properly should be gone. >>

Good point about SSDs.

<< I'm not relying on strictly isolated memory databases per process, but I
need a way to keep things as they are. Correct me if I'm wrong: making
in-memory database contents visible to all sessions at all times, would only
mean to me to create a memory db with a unique name for each session, and
simply accessing only that one in that session. I could live with that. >>

Cool, thanks.

Tim Young
Elevate Software
www.elevatesoft.com
Mon, Jan 14 2013 9:07 AMPermanent Link

Tim Young [Elevate Software]

Elevate Software, Inc.

Avatar

Email timyoung@elevatesoft.com

Heiko,

<< But at some point the server process is terminated, right? Why not drop
the catalogs then? >>

They do get dropped then.  The issue isn't that they are *eventually*
removed from memory, but rather that they accumulate as clients
connect/disconnect sessions from different processes that have created
in-memory database tables.

<< And...what would happen if we had a shared memory database, a client
crashes and doesn't clean up "his" temporary tables? There would be an even
greater memory leak, wouldn't it? >>

Not if the client created the tables as proper temporary tables (CREATE
TEMPORARY TABLE vs. CREATE TABLE).  Also, if the client did create the
tables as normal tables (not temporary), it still wouldn't be what I would
call a "leak", rather just a bug in the client application code (unless
that's what the application wants).  The current situation is a proper leak,
in that normal operation of the client application code can cause memory
increases through no fault of the application code itself.

Tim Young
Elevate Software
www.elevatesoft.com
Mon, Jan 14 2013 9:07 AMPermanent Link

Tim Young [Elevate Software]

Elevate Software, Inc.

Avatar

Email timyoung@elevatesoft.com

Roy,

<< It started me thinking about the whole can of worms that could be. Maybe
Tim will need to introduce (as Barry thought existed) TEMPORARY tables for
the in-memory database as well. >>

Those already exist.

Tim Young
Elevate Software
www.elevatesoft.com
« Previous PagePage 2 of 3Next Page »
Jump to Page:  1 2 3
Image