Skip to content

Conversation

@magicalyak
Copy link
Contributor

@magicalyak magicalyak commented Oct 21, 2025

Summary

This PR fixes issue #748 where users experience storage errors with Django 4.2+ when using cloud storage backends (S3, Google Cloud Storage, etc.).

The Problem

Users reported errors like:

ValueError: Cannot determine path without bucket name.

When using Django's STORAGES configuration with OPTIONS (bucket names, credentials, etc.), sorl-thumbnail would fail because storage instances were created without these required OPTIONS.

Real-world impact: Projects like NetBox are blocked by this issue (netbox-community/netbox#20212), unable to use S3 storage with sorl-thumbnail until this fix is released.

Root Cause

While commit b348c63 added support for Django storage aliases in get_or_create_storage(), the serialization still used the backend class path instead of the alias. This meant:

  1. ✅ Create thumbnail → Uses storages["default"] with OPTIONS
  2. ❌ Serialize → Saves "storages.backends.s3.S3Storage" (class path)
  3. ❌ Deserialize → Calls get_module_class()() without OPTIONS → FAILS

As @panizza noted in issue #748:

"When the serialize_image_file method is called you serialize the storage path, not the storage alias... the deserialize_image_file retrieves the saved storage path and create a new storage without options, leading to errors."

The Solution

This PR completes the fix by ensuring storage aliases are serialized whenever possible:

  1. ✅ Create thumbnail → Uses storages["default"] with OPTIONS
  2. ✅ Serialize → Saves "default" (alias)
  3. ✅ Deserialize → Uses storages["default"] with OPTIONS → SUCCESS

Changes

  • Modified serialize_storage() to return the storage alias (e.g., "default") instead of the backend class path when the alias exists in Django's STORAGES configuration
  • Updated test expectations to match new serialization format
  • Rebased PR Refs #748 - Serialize storage alias whenever possible #780 by @claudep onto latest master to fix CI issues
  • Added imagemagick to GitHub workflow dependencies

Related Issues

Test Plan

  • All existing tests updated to reflect new serialization format
  • test_storage_serialize passes - directly validates storage alias serialization
  • Tests pass with Python 3.9 and 3.13
  • Code follows project style (ruff linting passes)
  • Coverage requirements met

Testing with Real S3 Configuration

This fix allows users to configure S3 storage like:

STORAGES = {
    "default": {
        "BACKEND": "storages.backends.s3.S3Storage",
        "OPTIONS": {
            "bucket_name": "my-bucket",
            "access_key": "...",
            "secret_key": "...",
            # ... other OPTIONS
        },
    },
}

And thumbnails will now correctly serialize/deserialize using the "default" alias, preserving all OPTIONS through the entire lifecycle.

@codecov
Copy link

codecov bot commented Oct 21, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 64.05%. Comparing base (0df1039) to head (b3c24c4).
⚠️ Report is 3 commits behind head on master.

❗ There is a different number of reports uploaded between BASE (0df1039) and HEAD (b3c24c4). Click for more details.

HEAD has 27 uploads less than BASE
Flag BASE (0df1039) HEAD (b3c24c4)
30 3
Additional details and impacted files
@@             Coverage Diff             @@
##           master     #788       +/-   ##
===========================================
- Coverage   74.07%   64.05%   -10.02%     
===========================================
  Files          26       26               
  Lines        1678     1686        +8     
===========================================
- Hits         1243     1080      -163     
- Misses        435      606      +171     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@magicalyak magicalyak force-pushed the fix-issue748-serialize-storage-alias branch from b3c24c4 to 6031c3a Compare October 28, 2025 11:30
@claudep
Copy link
Contributor

claudep commented Oct 30, 2025

Now that my PR for #748 was merged, maybe you can rebase so that we can push your commit?

@magicalyak magicalyak force-pushed the fix-issue748-serialize-storage-alias branch from 6031c3a to a940ba0 Compare November 2, 2025 00:43
@magicalyak
Copy link
Contributor Author

Now that my PR for #748 was merged, maybe you can rebase so that we can push your commit?

Looks like the only change this adds is for the CHANGES.rst so I'm thinking I can close this. Thanks so much for adding the changes, I really appreciate it! @claudep

@magicalyak magicalyak closed this Nov 2, 2025
@magicalyak magicalyak reopened this Nov 2, 2025
@magicalyak
Copy link
Contributor Author

@claudep not sure it's work a PR just for that but feel free to close if not (or let me know and I will).

Copy link
Contributor

@claudep claudep left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Documentation is precious, too!

@claudep claudep merged commit 9a0cebe into jazzband:master Nov 2, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Thumbnails don't work with S3 storage django 4.2 - storage options (problem with s3 and template tag)

2 participants