Fix pickle recursion

README.md

@@ -1,3 +1,81 @@
+# 💉✅ Django Unicorn Loosened
+
+This is a repository of a customized version of [Django Unicorn](https://github.com/adamghill/django-unicorn) package.
+
+_Loosened_ version of package is by default functionally the same as the official package. Extended stuff that were added are controllable with settings/feature flags.
+
+This repository is intended to sync periodically with `adamghill` repository in order to be up to date with the latest changes.
+
+Releasing as an installable package is planned only for a private repo for now, and version numbers will follow an official one with one extra dotted number. In the example, if there is an official version of `0.50.0`, the tag that follows this version here will be `0.50.0.1`.
+
+## Additional settings
+
+Available additional settings that can be set to `UNICORN` dict in settings.py which are not part of official package.
+
+- `USE_CSRF_TOKEN` - default: `True` - If set to `False`, unicorn does not check or send `csrf` token value so `{% csrf_token %}` is not mandatory in the templates. This is added due the fact to additional page caching system like `Varnish` does not operate effective if `Cookie` value is present in `Vary` header.
+- `CHECK_CHECKSUM_MATCH` - default: `True` - If set to `False`, `unicorn` does not perform data checksum check on each request.
+
+## Deployment
+
+1. Add your repository to poetry.config:
+  `poetry config repositories.myrepo http://to.my.repo`
+
+2. Publish package to your repository with `--build` flag
+  `poetry publish --build -r myrepo -u <myrepouser> -p <myrepopass>` 
+
+## Customization changelog
+
+### 0.61.0.1 - (2024-07-16)
+
+- Avoid recursion upon caching parent/child complex components with `pickle.dumps`
+- Sync with main package, version `0.61.0`
+
+### 0.60.0.1 - (2024-03-29)
+
+- No customizations, just sync with main package.
+
+### 0.58.1.2 - (2024-01-10)
+
+- add optional `CHECK_CHECKSUM_MATCH` setting which is set by default to True. If turned off, `unicorn` does not perform data checksum check on each request.
+
+### 0.58.1.1 - (2024-01-10)
+
+- No customizations, just sync with main package.
+
+### 0.57.1.1 - (2023-11-10)
+
+- No customizations, just sync with main package.
+
+### 0.55.0.1 - (2023-09-15)
+
+- No customizations, just sync with main package.
+
+### 0.54.0.1 - (2023-08-31)
+
+- No customizations, just sync with main package.
+
+### 0.53.0.1 - (2023-08-08)
+
+- No customizations, just sync with main package.
+
+### 0.50.1.1 - (2023-05-23)
+
+- No customizations, just sync with main package.
+
+### 0.50.0.1 - (2023-05-11)
+
+- Add `USE_CSRF_TOKEN` (`useCsrfToken`) setting that if set to `False` (`false`) avoids CSRF token check. By default  `USE_CSRF_TOKEN` is set to `True`.
+- [views.__init__.py:message] - Added decorator `csrf_handle` that checks `USE_CSRF_TOKEN` setting and based on boolean applies `csrf_protect` or `csrf_exempt` decorator.
+- [templatetags/unicorn.py:unicorn_scripts] - Added `USE_CSRF_TOKEN` to return in order to use in templates
+- [templates/unicorn/scripts.html] - Translate `USE_CSRF_TOKEN` value into `useCsrfToken` javascript variable and pass it to `Unicorn.init`
+- [static/unicorn/js/unicorn.js:init] - apply `useCsrfToken` to `args` that are used latter in component.
+- [static/unicorn/js/component.js:Component] - add `useCsrfToken` to class instance in constructor
+- [static/unicorn/js/messageSender.js:send] - set `csrf` token to headers only if `useCsrfToken` is set to `true`.
+
+--------------------------------
+## Official documentation below
+--------------------------------
+
 <p align="center">
   <a href="https://www.django-unicorn.com/"><img src="https://github.com/adamghill/django-unicorn/raw/a98539b6e4b1123705559116a77e63eea7e2b8d0/img/unicorn-logo.png" alt="django-unicorn logo" height="200"/></a>
 </p>

django_unicorn/cacher.py

@@ -86,6 +86,10 @@ def __enter__(self):
                 raise UnicornCacheError(
                     f"Cannot cache component '{type(component)}' because it is not picklable: {type(e)}: {e}"
                 ) from e
+            except RecursionError as e:
+                logger.warning(
+                    f"Cannot cache component '{type(component)}' because it is not picklable: {type(e)}: {e}"
+                )
 
         return self
 

django_unicorn/components/unicorn_template_response.py

@@ -247,6 +247,11 @@ def render(self):
                     for child in descendant.children:
                         init_script = f"{init_script} {child._init_script}"
                         json_tags.append(child._json_tag)
+                        # We need to delete this property here as it can cause RecursionError
+                        # when pickling child component. Tag element has previous_sibling
+                        # and next_sibling which would also be pickled and if they are big,
+                        # cause RecursionError
+                        del child._json_tag
                         descendants.append(child)
 
                 script_tag = soup.new_tag("script")

django_unicorn/static/unicorn/js/component.js

@@ -25,6 +25,7 @@ export class Component {
     this.key = args.key;
     this.messageUrl = args.messageUrl;
     this.csrfTokenHeaderName = args.csrfTokenHeaderName;
+    this.useCsrfToken = args.useCsrfToken;
     this.csrfTokenCookieName = args.csrfTokenCookieName;
     this.hash = args.hash;
     this.data = args.data || {};

django_unicorn/static/unicorn/js/messageSender.js

@@ -37,7 +37,18 @@ export function send(component, callback) {
     Accept: "application/json",
     "X-Requested-With": "XMLHttpRequest",
   };
-  headers[component.csrfTokenHeaderName] = getCsrfToken(component);
+  /**
+   * Override Reason: 
+      Optionally ignore `CSRF` check for effective varnish caching 
+      remove `Cookie` from `Vary` header.
+   */
+  if (component.useCsrfToken) {
+    try {
+      headers[component.csrfTokenHeaderName] = getCsrfToken(component);
+    } catch (err) {
+      console.error(err);
+    }
+  }
 
   fetch(component.syncUrl, {
     method: "POST",

django_unicorn/static/unicorn/js/unicorn.js

@@ -5,6 +5,7 @@ import { getMorpher } from "./morpher.js";
 
 let messageUrl = "";
 let csrfTokenHeaderName = "X-CSRFToken";
+let useCsrfToken = true;
 let csrfTokenCookieName = "csrftoken";
 let morpher;
 
@@ -17,7 +18,8 @@ export function init(
   _messageUrl,
   _csrfTokenHeaderName,
   _csrfTokenCookieName,
-  _morpherSettings
+  _morpherSettings,
+  _useCsrfToken
 ) {
   messageUrl = _messageUrl;
 
@@ -27,6 +29,13 @@ export function init(
     csrfTokenHeaderName = _csrfTokenHeaderName;
   }
 
+  /**
+   * Extend Reason: 
+      Optionally ignore `CSRF` check for effective varnish caching 
+      remove `Cookie` from `Vary` header by setting `useCsrfToken` to false
+   */
+  if (_useCsrfToken !== undefined) useCsrfToken = _useCsrfToken;
+
   if (hasValue(_csrfTokenCookieName)) {
     csrfTokenCookieName = _csrfTokenCookieName;
   }
@@ -36,6 +45,7 @@ export function init(
     csrfTokenHeaderName,
     csrfTokenCookieName,
     morpher,
+    useCsrfToken
   };
 }
 
@@ -47,6 +57,7 @@ export function componentInit(args) {
   args.csrfTokenHeaderName = csrfTokenHeaderName;
   args.csrfTokenCookieName = csrfTokenCookieName;
   args.morpher = morpher;
+  args.useCsrfToken = useCsrfToken;
 
   const component = new Component(args);
   components[component.id] = component;

django_unicorn/templates/unicorn/scripts.html

@@ -2,16 +2,17 @@
 
 {{ MORPHER|json_script:"unicorn:settings:morpher" }}
 
+<script>
+  var useCsrfToken = "{{ USE_CSRF_TOKEN }}" == "True";
+</script>
 {% if MINIFIED %}
-<script src="{% static 'unicorn/js/unicorn.min.js' %}"></script>
-{% comment %} <script src="../static/unicorn/js/unicorn.min.js"></script> {% endcomment %}
+<script src="{% static 'unicorn/js/unicorn.min.js' %}?bust=20240329"></script>
 
 <script>
   const url = "{% url 'django_unicorn:message' %}";
   const morpherSettings = JSON.parse(document.getElementById("unicorn:settings:morpher").textContent);
 
-  Unicorn.init(url, "{{ CSRF_HEADER_NAME }}", "{{ CSRF_COOKIE_NAME }}", morpherSettings);
-
+  Unicorn.init(url, "{{ CSRF_HEADER_NAME }}", "{{ CSRF_COOKIE_NAME }}", morpherSettings, useCsrfToken);
 </script>
 {% else %}
 <script type="module">
@@ -24,6 +25,6 @@
   const url = "{% url 'django_unicorn:message' %}";
   const morpherSettings = JSON.parse(document.getElementById("unicorn:settings:morpher").textContent);
 
-  Unicorn.init(url, "{{ CSRF_HEADER_NAME }}", "{{ CSRF_COOKIE_NAME }}", morpherSettings);
+  Unicorn.init(url, "{{ CSRF_HEADER_NAME }}", "{{ CSRF_COOKIE_NAME }}", morpherSettings, useCsrfToken);
 </script>
 {% endif %}

django_unicorn/templatetags/unicorn.py

@@ -31,6 +31,7 @@ def unicorn_scripts():
 
     return {
         "MINIFIED": get_setting("MINIFIED", not settings.DEBUG),
+        "USE_CSRF_TOKEN": get_setting("USE_CSRF_TOKEN", True),
         "CSRF_HEADER_NAME": csrf_header_name,
         "CSRF_COOKIE_NAME": csrf_cookie_name,
         "MORPHER": get_morpher_settings(),

django_unicorn/views/__init__.py

@@ -11,7 +11,7 @@
 from django.http import HttpRequest, JsonResponse
 from django.http.response import HttpResponseNotModified
 from django.utils.safestring import mark_safe
-from django.views.decorators.csrf import csrf_protect, ensure_csrf_cookie
+from django.views.decorators.csrf import csrf_protect, csrf_exempt, ensure_csrf_cookie
 from django.views.decorators.http import require_POST
 
 from django_unicorn.components import UnicornView
@@ -23,6 +23,7 @@
 )
 from django_unicorn.serializer import loads
 from django_unicorn.settings import (
+    get_setting,
     get_cache_alias,
     get_serial_enabled,
     get_serial_timeout,
@@ -523,13 +524,27 @@ def _handle_queued_component_requests(request: HttpRequest, queue_cache_key) ->
     return first_json_result
 
 
+def csrf_handle(func):
+    """
+    In case if `USE_CSRF_TOKEN` is set to False
+    CSRF token is not required.
+    """
+    if get_setting("USE_CSRF_TOKEN", True):
+        return ensure_csrf_cookie(csrf_protect(func))
+    return csrf_exempt(func)
+
+
 @timed
 @handle_error
-@ensure_csrf_cookie
-@csrf_protect
+@csrf_handle
 @require_POST
 def message(request: HttpRequest, component_name: Optional[str] = None) -> JsonResponse:
     """
+    Overwrite Reason:
+        Optionally Ignore `CSRF` check for effective varnish caching remove `Cookie` from `Vary` header.
+        Use `csrf_handle` decorator to optionally enable requests without `CSRF` token,
+        based on `USE_CSRF_TOKEN` setting.
+    ----
     Endpoint that instantiates the component and does the correct action
     (set an attribute or call a method) depending on the JSON payload in the body.
 

django_unicorn/views/objects.py

@@ -5,6 +5,7 @@
 from django_unicorn.components import HashUpdate, LocationUpdate, PollUpdate
 from django_unicorn.errors import UnicornViewError
 from django_unicorn.serializer import JSONDecodeError, dumps, loads
+from django_unicorn.settings import get_setting
 from django_unicorn.utils import generate_checksum, is_int
 
 logger = logging.getLogger(__name__)
@@ -88,6 +89,9 @@ def validate_checksum(self):
         """
         Validates that the checksum in the request matches the data.
 
+        If `CHECK_CHECKSUM_MATCH` is set to False,
+        checking of matching checksum is avoided.
+
         Returns:
             Raises `AssertionError` if the checksums don't match.
         """
@@ -97,11 +101,14 @@ def validate_checksum(self):
             # TODO: Raise specific exception
             raise AssertionError("Missing checksum")
 
-        generated_checksum = generate_checksum(self.data)
+        check_checksum_match = get_setting("CHECK_CHECKSUM_MATCH", True)
 
-        if checksum != generated_checksum:
-            # TODO: Raise specific exception
-            raise AssertionError("Checksum does not match")
+        if check_checksum_match:
+            generated_checksum = generate_checksum(self.data)
+
+            if checksum != generated_checksum:
+                # TODO: Raise specific exception
+                raise AssertionError("Checksum does not match")
 
 
 class Return:

pyproject.toml

@@ -1,20 +1,23 @@
 [tool.poetry]
-name = "django-unicorn"
-version = "0.61.0"
-description = "A magical full-stack framework for Django."
-authors = ["Adam Hill <unicorn@adamghill.com>"]
+name = "django-unicorn-loosened"
+version = "0.61.0.1"
+description = "An alternate version of a magical full-stack framework for Django."
+authors = ["Adam Hill <unicorn@adamghill.com>", "Styria Digital Development <digital.development@styria.dev>"]
 license = "MIT"
 readme = "README.md"
-repository = "https://github.com/adamghill/django-unicorn/"
+repository = "https://github.com/Styria-Digital/django-unicorn"
 homepage = "https://www.django-unicorn.com"
 documentation = "https://www.django-unicorn.com/docs/"
 keywords = ["django", "python", "javascript", "fullstack"]
+packages = [
+    { include = "django_unicorn" }
+]
 
 [tool.poetry.urls]
 "Funding" = "https://github.com/sponsors/adamghill"
 
 [tool.poetry.dependencies]
-python = ">=3.10"
+python = ">=3.9"
 django = ">=2.2"
 beautifulsoup4 = ">=4.8.0"
 orjson = ">=3.6.0"