Stop Duplicate Django Imports with get_or_create
A supplier sends the same SKU again, this time with a new name and price. Should your import create a product, keep the existing details, or update them?
get_or_create() answers part of that question. Your lookup fields, database constraint, and update policy answer the rest.
Here is the practical pattern from my repeated-import guide, using a supplier product catalog.
Start with the product identity
A supplier’s SKU identifies a product within that supplier’s catalog. Another supplier can legitimately use the same SKU.
The identity is therefore supplier_code plus sku. The display name and price describe the product, but they can change without creating a new identity.
That distinction matters in a lookup. If you include a name that changed, Django may fail to find the existing row and attempt an insert. Without a constraint, that can produce a duplicate. With the right constraint, it can produce an integrity error instead.
Neither outcome is the import policy you intended.
Make the database enforce that identity
In supplier_import/models.py, define the boundary explicitly:
from django.db import models
class Product(models.Model):
supplier_code = models.CharField(max_length=30)
sku = models.CharField(max_length=40)
name = models.CharField(max_length=120)
price = models.DecimalField(max_digits=10, decimal_places=2)
class Meta:
constraints = [
models.UniqueConstraint(
fields=["supplier_code", "sku"],
name="supplier_import_unique_sku",
),
]Register the app in INSTALLED_APPS, then create and apply its migration:
python manage.py makemigrations supplier_import
python manage.py migrateThese identity fields are non-null. The constraint is immediate and unconditional, matching the lookup we will use.
For an existing table, inspect duplicate identities before adding the constraint. Resolve which record and relationships should survive rather than deleting data just to make a migration pass.
Put initial values in defaults
Create supplier_import/importer.py:
from decimal import Decimal
from .models import Product
def import_product(*, supplier_code, sku, name, price):
return Product.objects.get_or_create(
supplier_code=supplier_code,
sku=sku,
defaults={"name": name, "price": Decimal(price)},
)The lookup finds the product. defaults supplies fields only when Django creates it.
The return value is (product, created). When created is false, an existing row was found. It does not mean the new feed values were applied.
Get Django Wiki’s stories in your inbox
Join Medium for free to get updates from this writer.
This is useful when staff edit product descriptions or prices locally and repeated imports should preserve those changes.
Pass decimal prices as strings, such as "12.50". Validate identifiers, lengths, price ranges, and supplier-specific normalization before calling this function. ORM creation helpers do not call full_clean() automatically.
Pick up my free Django cheatsheets and books at djangowiki.gumroad.com
Check the second import explicitly
In a fresh example database, run this in Django’s shell:
from decimal import Decimal
from supplier_import.importer import import_product
from supplier_import.models import Product
first, created = import_product(
supplier_code="ACME", sku="MUG-01",
name="Blue mug", price="12.50",
)
assert created
second, created = import_product(
supplier_code="ACME", sku="MUG-01",
name="Ocean mug", price="14.00",
)
assert not created
assert second.pk == first.pk
assert second.name == "Blue mug"
assert second.price == Decimal("12.50")
assert Product.objects.filter(supplier_code="ACME", sku="MUG-01").count() == 1The important result is one persisted identity with its original fields intact. If this product already exists before the demonstration, the first created assertion will not be true.
Use a separate test for a second supplier with the same SKU. That should create another row because the supplier is part of the identity.
Choose updates as a separate policy
If the supplier feed is authoritative for names and prices, add an explicit sync function to the same module:
def sync_product(*, supplier_code, sku, name, price):
return Product.objects.update_or_create(
supplier_code=supplier_code,
sku=sku,
defaults={"name": name, "price": Decimal(price)},
)Now a repeated import deliberately changes those fields:
from supplier_import.importer import sync_product
updated, created = sync_product(
supplier_code="ACME", sku="MUG-01",
name="Ocean mug", price="14.00",
)
assert not created
assert updated.pk == first.pk
assert updated.price == Decimal("14.00")The lookup stays the same. Only the overwrite policy changes. Naming the functions separately makes that decision visible to the code that calls them.
Keep the guarantee narrow
If a repeated import raises IntegrityError, compare the lookup with the constraint. A changing name in the lookup can hide the existing row while the supplier/SKU constraint still blocks insertion.
If it raises MultipleObjectsReturned, inspect existing duplicates and whether the intended constraint was applied.
Django’s get_or_create reference makes its concurrent-insert protection dependent on database uniqueness for the lookup fields. Sequential import tests do not simulate competing workers or every isolation scenario.
Also keep database identity separate from side effects. Finding or creating one product does not guarantee that an email or payment runs exactly once. Perform writes through an appropriate POST action or background import, rather than a GET page load.
Choose stable identity fields, enforce them in the database, and decide explicitly whether imports preserve or overwrite existing values.
