wetsuite.helpers.localdata.MsgpackKV

class documentation

class MsgpackKV(LocalKV):

View In Hierarchy

Like localKV but the value can be a nested python type (serialized via msgpack)

Will be a bit slower because it's doing that on the fly, but lets you more transparently store things like nested python dicts ...but only of primitive types, and not e.g. datetime.

Typing is fixed, to str:bytes

msgpack is used as a somewhat faster alternative to json and pickle (though that overhead barely matters for smaller structures)

Note that this does _not_ change how the meta table works.

Method __init__ value_type is ignored; I need to restructure this
Method get Note that unpickling could fail
Method iteritems Returns a generator that yields all items
Method itervalues Returns a generator that yields all values. If you wanted a list with all the values, use list( store.values )
Method put See LocalKV.put(). Unlike that, value is not checked for type, just serialized. Which can fail with an exception.

Inherited from LocalKV:

Method __contains__ will return whether the store contains a key
Method __enter__ supports use as a context manager
Method __exit__ supports use as a context manager - close()s on exit
Method __getitem__ (only meant to support ValuesView and Itemsview)
Method __iter__ Using this object as an iterator yields its keys (equivalent to .iterkeys())
Method __len__ Return the amount of entries in this store
Method __repr__ show useful representation
Method bytesize Returns the approximate amount of the contained data, in bytes (may be a few dozen kilobytes off, or more, because it counts in pages)
Method close Closes file if still open. Note that if there was a transaction still open, it will be rolled back, not committed.
Method commit commit changes - for when you use put() or delete() with commit=False to do things in a larger transaction
Method delete delete item by key.
Method estimate_waste Estimate how many bytes might be cleaned by a .vacuum()
Method items Returns an iteralble of all items. (a view with a len, rather than just a generator)
Method iterkeys Returns a generator that yields all keus If you wanted a list with all keys, use list( store.keys() )
Method keys Returns an iterable of all keys. (a view with a len, rather than just a generator)
Method random_choice Returns a single (key, value) item from the store, selected randomly.
Method random_keys Returns a amount of keys in a list, selected randomly. Can be faster/cheaper to do than random_sample When the values are large
Method random_sample Returns an amount of [(key, value), ...] list from the store, selected randomly.
Method random_sample_generator A generator that yields one (key,value) tuple at a time, intended to avoid materializing all values before we return.
Method random_values Returns a amount of values in a list, selected randomly.
Method random_values_generator A generator that yields one value at a time, intended to avoid materializing all values before we return.
Method rollback roll back changes
Method summary Gives the byte size, and optionally the number of items and average size
Method truncate remove all kv entries. If we were still in a transaction, we roll that back first
Method vacuum After a lot of deletes you could compact the store with vacuum(). WARNING: rewrites the entire file, so the more data you store, the longer this takes. And it may make no difference - you probably want to check estimate_waste() first...
Method values Returns an iterable of all values. (a view with a len, rather than just a generator)
Instance Variable conn connection to the sqlite database that we set up
Instance Variable key_type the key type you have set
Instance Variable path the path we opened (after resolving)
Instance Variable read_only whether we have told ourselves to treat this as read-only. That _should_ also make it hard for _us_ to be the cause of leaving the database in a locked state.
Instance Variable value_type the value type you have set
Method _checktype_key checks a value according to the key_type you handed into the constructor
Method _checktype_value checks a value according to the value_type you handed into the constructor
Method _delete_meta For internal use, preferably don't use. See also _get_meta(), _delete_meta(). Note this does an implicit commit()
Method _get_meta For internal use, preferably don't use.
Method _open Open the path previously set by init. This function could probably be merged into init, it was separated mostly with the idea that we could keep it closed when not using it.
Method _put_meta For internal use, preferably don't use. See also _get_meta(), _delete_meta(). Note this does an implicit commit()
Instance Variable _in_transaction Undocumented
def __init__(self, path, key_type=str, value_type=None, read_only=False):
overrides wetsuite.helpers.localdata.LocalKV.__init__

value_type is ignored; I need to restructure this

def get(self, key, missing_as_none=False):
overrides wetsuite.helpers.localdata.LocalKV.get

Note that unpickling could fail

Note also that you may wish to never explicitly store just None, and/or never use missing_as_none, unless you like ambiguity.

Parameters
key:strUndocumented
missing_as_noneUndocumented
def iteritems(self):
overrides wetsuite.helpers.localdata.LocalKV.iteritems

Returns a generator that yields all items

def itervalues(self):
overrides wetsuite.helpers.localdata.LocalKV.itervalues

Returns a generator that yields all values. If you wanted a list with all the values, use list( store.values )

def put(self, key, value, commit=True):
overrides wetsuite.helpers.localdata.LocalKV.put

See LocalKV.put(). Unlike that, value is not checked for type, just serialized. Which can fail with an exception.

Parameters
key:strUndocumented
valueUndocumented
commit:boolUndocumented