cursor leak

[eugwne]

leak example:

do $$
local cmd = 'select generate_series(1,150000)'
local plan = server.prepare(cmd)
local cur = plan:getcursor(nil, true)
while true do
    local r = cur:fetch(1)  
    if r == nil then break end
end
cur:close()
$$ language pllua

[jarwin123]

yes，memory leak！i have the same problem！have some new news to slove this question？

[anthony-hz]

Seens that this issue is not resolved, how can we get help?

[eugwne]

I was thinking about rewriting it, but I'm not so enthusiastic because server.rows works fine for me and I don't know how to rewrite it fast

[golgote]

Maybe it's just a missing SPI_freetuptable...

[eugwne]

I tried, but after that some tests were broken. It looks like this:

local r = cur:fetch(1)  
local lost_value = r.column
r = cur:fetch(1)  -- SPI_freetuptable can destroy lost_value 

[jarwin123]

i use this through SPI_xxx function,my memoy leak faster. i want to know why?

[jarwin123]

@eugwne ，you say " server.rows works fine for you",how to use it,can you tell me your lua script?thank you!

[eugwne]

do $$
for row in server.rows("select 42 as column") do
    print(row.column)
end
$$ language pllua

[anthony-hz]

Is there any plan to fix this bug?

[jarwin123]

@eugwne,
if i test it use this :
CREATE or replace FUNCTION test() RETURNS void AS $$
local cmd="select no as column from test" --no is text type.
for row in server.rows(cmd) do
print(row.column)
end
$$ LANGUAGE plluau;
it cause segment fault.

Segmentation fault (PID 24109)
version: PipelineDB at revision on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-52), 64-bit
backtrace:
pipeline-server: postgres online [local] SELECT(debug_segfault+0x22)[0x70d816]
/lib64/libc.so.6[0x39dee302d0]
pipeline-server: postgres online [local] SELECT(text_to_cstring+0x12e)[0x9493f5]
pipeline-server: postgres online [local] SELECT(textout+0x21)[0x94a4a7]
pipeline-server: postgres online [local] SELECT(DirectFunctionCall1Coll+0x7e)[0x989c66]
/usr/local/pipeline/lib/pipelinedb/pllua.so(luaP_pushdatum+0x1cb)[0x2b96abc8b3ee]
/usr/local/pipeline/lib/pipelinedb/pllua.so[0x2b96abc8fc73]
/usr/lib64/liblua.so[0x2b96abec0bb4]
/usr/lib64/liblua.so[0x2b96abec0fad]
/usr/lib64/liblua.so[0x2b96abec9202]
/usr/lib64/liblua.so[0x2b96abeca0e1]
/usr/lib64/liblua.so[0x2b96abec0ffe]
/usr/lib64/liblua.so[0x2b96abec0767]
/usr/lib64/liblua.so[0x2b96abec07e5]
/usr/lib64/liblua.so(lua_pcall+0x55)[0x2b96abebc905]
/usr/local/pipeline/lib/pipelinedb/pllua.so(luaP_callhandler+0x724)[0x2b96abc8d85e]
/usr/local/pipeline/lib/pipelinedb/pllua.so(pllua_call_handler+0x27)[0x2b96abc875f5]
pipeline-server: postgres online [local] SELECT[0x686ff2]
pipeline-server: postgres online [local] SELECT[0x6878cc]
pipeline-server: postgres online [local] SELECT[0x68e10c]
pipeline-server: postgres online [local] SELECT(ExecProject+0x316)[0x68e6f5]
pipeline-server: postgres online [local] SELECT(ExecResult+0x123)[0x6abf27]
pipeline-server: postgres online [local] SELECT(ExecProcNode+0x29f)[0x6832ba]
pipeline-server: postgres online [local] SELECT(ExecutePlan+0x5f)[0x6805b7]
pipeline-server: postgres online [local] SELECT(standard_ExecutorRun+0x170)[0x67e3a5]
pipeline-server: postgres online [local] SELECT(ExecutorRun+0x45)[0x67e233]
pipeline-server: postgres online [local] SELECT[0x82b8f4]
pipeline-server: postgres online [local] SELECT(PortalRun+0x209)[0x82b58e]
pipeline-server: postgres online [local] SELECT[0x8258b8]
pipeline-server: postgres online [local] SELECT(PostgresMain+0x6fd)[0x8298e8]
pipeline-server: postgres online [local] SELECT[0x7aa495]
pipeline-server: postgres online [local] SELECT[0x7a9b26]

why?

[eugwne]

postgres compiled with enable-debug?

[jarwin123]

yes，i add enable-debug option。

[eugwne]

postgres fills data with a garbage when debug enabled, pllua makes no copy and gets garbage

[jarwin123]

but i just exec select no as column from test no is text type，this column have value not garbage and not NULL。

[jarwin123]

i cann't understand your answer? when does postgres fill data with a garbage?

[eugwne]

in pg it looks like this :
context = createMemoryContext(...)
data = palloc(...)
deleteContext(data)
if enable-debug is on then data filled by garbage
if enable-debug is off data has old values, but next createMemoryContext can take previous context and overwrite data

[jarwin123]

if i add enable-debug option, how to fix this bug?

[eugwne]

one way is to make always copy values from pg to lua, another - make lua gc and pg memory contexts work together

[anthony-hz]

Why not just always copy values from pg to lua.

[jmealo]

@eugwne Kudos on all your work. I'm glad to see this getting some love. I've been reading through the issues. To be clear, this is a low-level binding when compared to PL/V8 and it's helpful to know a bit about luajit and pg (and read the code/examples) if we hit issues?