Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

how to get all sublevel db #238

Open
williamnie opened this issue Jul 6, 2023 · 2 comments
Open

how to get all sublevel db #238

williamnie opened this issue Jul 6, 2023 · 2 comments
Labels
question Support or open question(s)

Comments

@williamnie
Copy link

I want to directly retrieve all sublevels through the Level API, but I couldn't find any relevant APIs. Instead of using a method that involves reading all keys and looping through them to check if they contain certain characters, I think it would be more efficient to store a small index when creating the sublevel.

@fedd
Copy link

fedd commented Oct 28, 2023

I'm seconding this question.

In a relational database we would group by a part of a compound key to see if a category of entries is present.

sublevel key value
=========================
example 19 [object]
example 20 [object]
another a [object]

select sublevel from kvstore group by sublevel
This can still fully scan the table, but can also try to get the index stats instead
select sublevel, count(*) from kvstore group by sublevel

In a key-value db when we concatenate category into a key, it would be nice if the sublevels are tracked somehow

key value
=======================
!example!19 [object]
!example!20 [object]
!another!a [object]

How do we get all of the existing sublevels?

@vweevers
Copy link
Member

You can use the database itself as an index and query it with a skip scan to discover sublevels. Roughly like so (not fully tested):

const { AbstractSublevel } = require('abstract-level')

// Discover sublevels using a skip scan
async function sublevels (db, options) {
  const separator = AbstractSublevel.defaults(options).separator
  const upperBound = String.fromCharCode(separator.charCodeAt(0) + 1)
  const iterator = db.keys({ keyEncoding: 'utf8' })
  const names = []

  try {
    let target = separator

    while (true) {
      iterator.seek(target)
      const key = await iterator.next()

      // Stop if no sublevel prefix is found
      if (key === undefined || key[0] !== separator) {
        break
      }

      // Check if key has a complete sublevel prefix, e.g. '!a!'
      const i = key.indexOf(separator, 1)
      if (i === -1) break

      // Extract name, e.g. 'a'
      const name = key.slice(1, i)
      names.push(name)

      // Seek to next sublevel that follows '!a"'
      target = separator + name + upperBound
    }
  } finally {
    await iterator.close()
  }

  return names
}

@vweevers vweevers added the question Support or open question(s) label Jan 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Support or open question(s)
Projects
None yet
Development

No branches or pull requests

3 participants