OverviewUniqueModel is provides a mechanism for application developers to define a list of property tuples to be kept unique on the Google App Engine Datastore.
See it in action (code)
Noteworthy FeaturesSQL-like robust uniqueness enforcement Able to enforce uniqueness of multiple property
... [More]
tuples for a single kind Uniqueness is enforced even on concurrent put() operations Virtually no programming overhead for normal usage (see example below) Support and FeedbackIf you decide to give UniqueModel a try, I'd love to hear from you. Please send me an e-mail (my address is located to the right) and submit any issues you find to the issue tracker.
If you need support, I can be reached by e-mail, or I can also be found lingering in #appengine on irc.freenode.net under the nick IAB.
Simple ExampleAs an example, lets say I'm writing a simple game in which players interact on a rectangular 2D map. To represent a player's location on the map, I create a structure called locations in which I store the x and y value of the player's position, and the ID of an active player.
I'm scared of ghosts and things that can be in multiple places at the same time. I know, it's an irrational fear, but it's my game dammit! In order to keep more than one player from occupying the same position on the map, and therefore keep the ghosts at bay, I need to make sure that a point is represented exactly 0 or 1 times in my locations structure. Also in order to be sure nobody occupies more than one position, I need to be sure that an active player_id is represented exactly 0 (an active player can be dead) or 1 times. In scientific terms, the property tuple (x,y) and the single property player_id must each be unique within the set of all locations
In a typical SQL schema definition script this would be accomplished like so:
CREATE TABLE locations(
x integer,
y integer,
player_id integer references active_players(player_id),
unique(x,y),
unique(id),
);There is no similar way of enforcing uniqueness in the App Engine Datastore. Fortunately to alleviate this problem, we can use UniqueModel:
from google.appengine.ext import db
import burns
class Locations(burns.UniqueModel):
x = db.IntegerProperty()
y = db.IntegerProperty()
player = db.ReferenceProperty(reference_class=ActivePlayer)
_uniques = set([
(x,y),
(player,) #note that single properties must be stored as tuple
])Just like with the SQL schema definition, if an insert or update violates the uniqueness constraint placed on (x,y) or player, an exception (UniqueConstraintViolatedError) is raised. No ghost-like or omni-present players to deal with!
Under the HoodIn a nutshell, UniqueModel works by creating an MD5 hash of a property tuple to be kept unique and storing it on the datastore. When put() is called on an instance of UniqueModel the following happens:
UniqueModel calculates a hash of each property tuple to be kept unique In a transaction, UniqueModel checks for the existence of these hashes in the datastore and does the following: If any of the hashes exist in the datastore, the transaction raises an exception Otherwise, each of the hashes are stored with a reference back to the UniqueModel that they represent. If the transaction raised an exception, UniqueModel calls delete on itself to undo the previous put() and then raises the exception again If no exception was raised, db.Model.put() is called and UniqueModel.put() returns normally LimitationsThere are a few limitations imposed by UniqueModel that the application developer should be aware of.
Stores many additional EntitiesThe uniqueness of each kind is modeled as entities of kind UniquePropertyInstance. Given n entities of a specific kind that extends UniqueModel, n*m entities of kind UniquePropertyInstance must exist to model the uniqueness of that kind, where m is equal to the number of property tuples to be kept unique for the kind in question.
Uniqueness is Modeled as an Entity GroupIn order to verify uniqueness in a method that is safe from concurrent put() operations, the uniquness must be tested and set in an atomic operation. The only way to do this with the App Engine datastore is to use a transaction. However, transactions have the unfortunate restriction that they can only operate on a single entity group. That means that in order to check the uniqueness for an entire kind, the entities that model the uniqueness for that kind must have a common parent.
Because of all this, UniqueModel stores an entity of kind UniquePropertyInstanceParent for each kind that extends UniqueModel. This entity serves as the root of the entity group for a particular kind that extends UniqueModel.
Therefore, for each kind extending UniqueModel, an entity group of size n*m+1 must exist to ensure uniqueness on concurrent put() operations, where n is equal to the number of entities of the kind and m is equal to the number of property tuples to be kept unique for the kind.
Uniqueness is Verified in a TransactionAs stated above, it is necessary to check and submit the uniqueness of an entity within a transaction. Unfortunately, since transactions on the datastore do not nest UniqueModel.put() cannot be called from within a transaction.
Uniqueness is Checked with MD5 HashesTo create the hash of a property tuple, each member of the tuple's __str__ method is called and concatenated. The resulting str is then passed to hashlib to be turned into an MD5 sum. This imposes the following problems:
Extra CPU usage on put() (see http://uniqueificate.appspot.com,code for an example) Only works for properties with __str__ methods that accurately represent the property value Possibility (although very low likelihood) of forced collisions [Less]
Copyright
©
2013
Black Duck Software, Inc.
and its contributors, Some Rights Reserved. Unless otherwise marked, this work is licensed under a
Creative Commons Attribution 3.0 Unported License
. Ohloh
®
and the Ohloh logo are trademarks of
Black Duck Software, Inc.
in the United States and/or other jurisdictions. All other trademarks are the property of their respective holders.