Arches 8.0.0 Release Notes -------------------------- ### Major enhancements - Adds draft_graphs and the ability to update Graphs without unpublishing. [#9613](https://github.com/archesproject/arches/issues/9613) - Adds `ResourceInstanceLifecycle`s and `ResourceInstanceLifecycleState`s. [#11042](https://github.com/archesproject/arches/issues/11042) - Removes RequireJS dependency. This reduces webpack bundle size and build time, and adds cache invalidation [#11902](https://github.com/archesproject/arches/issues/11902) - Add token-based CSS theming [#11262](https://github.com/archesproject/arches/issues/11262) - Support Python 3.13 [#11550](https://github.com/archesproject/arches/pull/11550) - Support Django 5.2 [#11797](https://github.com/archesproject/arches/pull/11797) ### Performance improvements - Improve indexing and bulk deletion performance [#11382](https://github.com/archesproject/arches/issues/11382) [#11553](https://github.com/archesproject/arches/issues/11553) - Improve serialization performance [#11962](https://github.com/archesproject/arches/pull/11962) - The following methods or functions in the permissions framework now take an optional ``resource`` keyword argument to avoid refetching an instance that a caller may already have: - `user_can_read_resource()` - `user_can_edit_resource()` - `user_can_delete_resource()` - `check_resource_instance_permissions()` - Reduce queries in card API [#11534](https://github.com/archesproject/arches/pull/11534) - Reduce queries in resource report view [#11637](https://github.com/archesproject/arches/pull/11637) - Add index to edit_log table timestamps to resolve timeouts on instances with lots of resource changes [#7853](https://github.com/archesproject/arches/issues/7853) - Removes webpack requirement to communicate with Django server during build process [#12084](https://github.com/archesproject/arches/issues/12084) ### Additional highlights - Add session-based REST APIs for login, logout [#11261](https://github.com/archesproject/arches/issues/11261) - Auth views now filter out passwords from error reports when running in production [#11652](https://github.com/archesproject/arches/issues/11652) - Default setting for `LOGGING` now logs errors from web requests [#11812](https://github.com/archesproject/arches/issues/11812) - Improve handling of longer model names [#11317](https://github.com/archesproject/arches/issues/11317) - New column `NodeGroup.grouping_node`: one-to-one field to the grouping node [#11613](https://github.com/archesproject/arches/issues/11613) - Constrain `CardXNodeXWidget` instances to one per node [#11958](https://github.com/archesproject/arches/issues/11958) - Support more expressive plugin URLs [#11320](https://github.com/archesproject/arches/issues/11320) - Make node aliases not nullable [#10437](https://github.com/archesproject/arches/issues/10437) - Make node field on graph not nullable [#12103](https://github.com/archesproject/arches/issues/12103) - Make `LoadStaging.nodegroup` nullable [#10887](https://github.com/archesproject/arches/issues/10887) - Add alt text on image placeholder [#10455](https://github.com/archesproject/arches/issues/10455) - Tile functions no longer catch TypeError [#11805](https://github.com/archesproject/arches/issues/11805) - Fix querying on related model's I18n_TextField [#11800](https://github.com/archesproject/arches/issues/11800) - Remove module-level query in URL datatype [#11821](https://github.com/archesproject/arches/issues/11821) - Make failed login alert message dismissible [#11767](https://github.com/archesproject/arches/issues/11767) - Concepts API no longer responds with empty body for error conditions [#11519](https://github.com/archesproject/arches/issues/11519) - Generates static `urls.json` file for frontend consumption [#11567](https://github.com/archesproject/arches/issues/11567) - Restore support for `.create()` on Django model managers [#10958](https://github.com/archesproject/arches/issues/10958) - Add default ordering on additional models [#11873](https://github.com/archesproject/arches/issues/11873) - Enforce single default language [#12027](https://github.com/archesproject/arches/pull/12027) - Removes sample index from new projects, updates test coverage behavior [#11591](https://github.com/archesproject/arches/issues/11519) - Creates a search filter to search for a list of ids [#11579](https://github.com/archesproject/arches/issues/11579) - Fix HTTP return code for tile validation and cardinality errors [#11803](https://github.com/archesproject/arches/issues/11803) - Add annotation count view in IIIF viewer [#11862](https://github.com/archesproject/arches/pull/11862) - Fix spatial view regex validator message [#11849](https://github.com/archesproject/arches/issues/11849) - Make `principaluser` read-only in the Django model definition [#11882](https://github.com/archesproject/arches/issues/11882) - Add system dark mode detection for Vue apps [#11624](https://github.com/archesproject/arches/issues/11624) - Make number datatype node values searchable in the main search [#11619](https://github.com/archesproject/arches/issues/11619) - Fix node sortorder in reorder nodes API response [#11651](https://github.com/archesproject/arches/issues/11651) - Prevent navigation to a new browser tab when clicking Manage link in index.htm [#11635](https://github.com/archesproject/arches/issues/11635) - Prevent `FILENAME_GENERATOR` changes from generating migrations [#12008](https://github.com/archesproject/arches/issues/12008) - Remove brittle guard against multiple calls to generate frontend configuration [#11701](https://github.com/archesproject/arches/issues/11701) - Mitigate the tendency of JSON APIs to respond with HTML under error conditions [#11722](https://github.com/archesproject/arches/pull/11722) - Add support for tile sort order to the bulk data manager [#11638](https://github.com/archesproject/arches/pull/11638) - Show the loading UX immediately when the user imports a resource model [#10540](https://github.com/archesproject/arches/issues/10540) - Improve default packaging manifest [#11828](https://github.com/archesproject/arches/issues/11828) - Migrate dev dependencies to dependency-groups [#12038](https://github.com/archesproject/arches/issues/12038) - Improve `ResourceXResource` field and related names [#11869](https://github.com/archesproject/arches/issues/11869) - Fix issue that produced an error when cycling between the dropdown options in Advanced search [#11442](https://github.com/archesproject/arches/issues/11442) - Add message when copying geojson url to clipboard [#11076](https://github.com/archesproject/arches/issues/11076) - Hide "view report" button in Arches system settings [#11659](https://github.com/archesproject/arches/issues/11659) - Updates github workflows postgis image to version 14-3.4 to match Arches core [#11833](https://github.com/archesproject/arches/issues/11833) - Allow datatypes to transform their default values for use on the front-end[#11904](https://github.com/archesproject/arches/issues/11904) & [#11944](https://github.com/archesproject/arches/issues/11944) - Add validation error for geometry node used in spatial view attributes [#11888](https://github.com/archesproject/arches/issues/11888) - Add source mapping to webpack development configuration [#11906](https://github.com/archesproject/arches/issues/11906) - Update the graph model to make the slug required [#11799](https://github.com/archesproject/arches/issues/11799) - Removes `arches_project` management commands, replacing them with `arches_admin` [#11574](https://github.com/archesproject/arches/issues/11574) - Add UserPreference model for storing custom JSON configuration for a user [#11842](https://github.com/archesproject/arches/issues/11842) - Give ETL module permissions to the dev user [#12016](https://github.com/archesproject/arches/pull/12016) - Update the ETL status page to display only the status from the current user [#12017](https://github.com/archesproject/arches/issues/12017) - New option for resource-instance datatype advanced search [#11977](https://github.com/archesproject/arches/pull/11977) ### Dependency changes ``` Python: Upgraded: Django: >=5.2.3, <6.0.0 celery: 5.5.0 django-celery-results: 2.6.0 django-cors-headers: 4.7.0 django-oauth-toolkit: 3.0.1 django-recaptcha: 4.1.0 django-revproxy: 0.13.0 django-webpack-loader: 3.1.1 psycopg2: 2.9.10 python-memcached: 1.62 openpyxl: 3.1.5 requests: >=2.32.3 requests-oauthlib: 2.0.0 setuptools: >=77 Added: django-pgtrigger: 4.15.2 Removed: tomli JavaScript: Upgraded: primevue == 4.3.3 vue ^3.5.13 (dev dependencies): vitest ^3.1.2 Added: primeicons == 7.0.0 @primeuix/themes == 1.01 Removed: @primevue/themes regenerator-runtime requirejs requirejs-plugins requirejs-text (dev dependencies): vue-template-compiler ``` ### Breaking changes - The minimum supported version of Python is now 3.11. - The minimum supported version of GDAL is now 2.4 (along with GEOS 3.8). - The `TileModel.nodegroup` foreign key constraint is no longer enforced, since it can now store UUIDs referring to historical nodegroups that no longer exist. - Following a deprecation in 7.6.0, the `-o install` argument to `manage.py packages` has been removed. - `ensure_userprofile_exists()` was removed from the `Tile` model. - The `arches` object is now only available in frontend components that are mounted in templates that inherit from `base.htm`. If your project contains components that are mounted in templates that inherit directly from `base-root.htm`, they will need to be updated to use `generateArchesURL` and the settings API instead. Please refer to [Upgrading an Arches project](#upgrading-an-arches-project). - The following fields are no longer nullable. If you have custom SQL (or Python code that uses direct ORM operations to bypass model `save()`, etc.), you will need to set these fields directly on creation: - `Node.alias` - `Node.graph` - `TileModel.data` - `Graph.slug` - The following fields are now nullable: - `LoadStaging.nodegroup` - `TileModel.nodegroup` - `ResourceXResource` model field names changed (without altering database column names): - `resourceinstanceidfrom` -> `from_resource` - `resourceinstanceidto` -> `to_resource` - `resourceinstancefrom_graphid` -> `from_resource_graph` - `resourceinstanceto_graphid` -> `to_resource_graph` - `nodeid` -> `node` - `tileid` -> `tile` - `ResourceXResource` model fields removed: - `datestarted` - `dateended` - The .relations file importer now accepts only the updated column headers: - `resourceinstanceidfrom` -> `from_resource` - `resourceinstanceidto` -> `to_resource` - `resourceinstancefrom_graphid` -> `from_resource_graph` - `resourceinstanceto_graphid` -> `to_resource_graph` - `nodeid` -> `node` - `tileid` -> `tile` - The following fields implemented [`related_name`](https://docs.djangoproject.com/en/stable/ref/models/fields/#django.db.models.ForeignKey.related_name) and [`related_query_name`](https://docs.djangoproject.com/en/stable/ref/models/fields/#django.db.models.ForeignKey.related_query_name) to improve upon the Django-supplied defaults: - `NodeGroup.parentnodegroup` - `TileModel.nodegroup` - `TileModel.parenttile` For instance, a tile query can prefetch its children with the more readable: ``` TileModel.objects.filter(...).prefetch_related("children") ``` Rather than: ``` TileModel.objects.filter(...).prefetch_related("tilemodel_set") # OLD NAME ``` In the event you had ORM queries traversing the "reverse" side of these relationships, (i.e. from one to many), make these changes: - From NodeGroup, traversing a child nodegroup: `nodegroup` -> `child` - From NodeGroup, collecting child nodegroups: `nodegroup_set` -> `children` - From NodeGroup, traversing a tile: `tilemodel` -> `tile` - From NodeGroup, collecting all tiles: `tilemodel_set` -> `tiles` - From TileModel, traversing a child tile: `tilemodel` -> `child` - From TileModel, collecting child tiles: `tilemodel_set` -> `children` - The related names on `ResourceXResource` were improved as follows: - From ResourceInstance, collecting relationships: - `resxres_resource_instance_ids_from` -> `from_resxres` - `resxres_resource_instance_ids_to` -> `to_resxres` - From Graph: - `resxres_resource_instance_fom_graph_id` -> `from_resxres` - `resxres_resource_instance_to_graph_id` -> `to_resxres` - From TileModel: `resxres_tile_id` -> `resxres_tile_id` - From Node: `resxres_node_id` -> `resxres` - The following models now have a default ordering by `sortorder` (or for ETLModule, `helpsortorder`): - CardModel (and Card) - CardXNodeXWidget - Node - Plugin - ETLModule In the event this ordering is not desired, chain an explicit ``order_by()`` with no arguments. - Tile cardinality errors raised when saving `TileModel` instances now raise `TileCardinalityError` rather than `ProgrammingError`. - `TileValidationError` and `TileCardinalityError` now subclass `django.core.exceptions.ValidationError`. - `GraphModel.get_published_graph()` no longer accepts the `raise_if_missing` argument. - `Graph.new()` has been deprecated in favor of `Graph.objects.create_graph()` - `WorkflowHistoryView` moved from `arches.app.views.workflow_history` to `arches.app.views.api`. - The format of JavaScript files has changed. Please refer to [Upgrading an Arches project](#upgrading-an-arches-project). - The format of how some images are loaded in template files has changed. Please refer to [Upgrading an Arches project](#upgrading-an-arches-project). - Following a deprecation in 7.6 `arches-project` management commands have been removed. - Django-oauth-toolkit has been upgraded to 3.0.1. ### Upgrading Arches 1. You must be upgraded to at least version 7.6.0 before proceeding. If you are on an earlier version, please refer to the upgrade process in the [Version 7.6.0 release notes](https://github.com/archesproject/arches/blob/dev/7.6.x/releases/7.6.0.md). 1. Be sure to backup your database before proceeding. 1. Upgrade to Arches 8.0.0: ``` pip install --upgrade pip pip install --upgrade arches==8.0.0 ``` ### Upgrading an Arches project 1. In pyproject.toml, make these updates: - Update `requires = ["setuptools", "setuptools-scm"]` to `requires = ["setuptools>=77", "setuptools-scm"]`. - Update `requires-python` to `">=3.11"` - Remove `"Programming Language :: Python :: 3.10"` - Add `"Programming Language :: Python :: 3.13"` - Remove `"Framework :: Django :: 4.2"` - Add `"Framework :: Django :: 5.2"` - Remove any classifier beginning with `"License :: "` - Replace the heading `[project.optional-dependencies]` with `[dependency-groups]` - Unless you have already published your project to PyPI, in the `name = ...` key replace underscores with hyphens to follow PEP 8 conventions. 1. Update `.gitignore`: - Remove: ``` .tsconfig-paths.json .frontend-configuration-settings.json ``` - Add: ``` frontend_configuration # Packaging build/ dist/ ``` 1. Replace any references to `arches-project` management commands with `arches-admin`. - `arches-project create` no longer exists, instead use `arches-admin startproject`. 1. In `MANIFEST.in`, remove `graft *`. Add exclusions for local settings files, e.g. `exclude /settings_local.py`. 1. In settings.py: - Add the following key to `DATABASES["default"]` to [potentially improve indexing performance](https://github.com/archesproject/arches/issues/11382): ``` DATABASES = { "default": { ... # other entries "OPTIONS": { "options": "-c cursor_tuple_fraction=1", }, } } ``` - Change the `"captcha"` entry in `INSTALLED_APPS` to reflect its new name and add pgtrigger: ``` INSTALLED_APPS = [ ... "django_recaptcha", # was "captcha" "pgtrigger", ... ] ``` - Remove "`django.contrib.admin"` from the first section of `INSTALLED_APPS`. - Add `"django.contrib.admin"` to the existing post addition to `INSTALLED_APPS`: ``` INSTALLED_APPS += ( "arches.app", "django.contrib.admin" ) ``` - In `LOGGING`, replace `"loggers"` with the following to begin logging failed web requests: ``` "loggers": { "arches": { "handlers": ["file", "console"], "level": "WARNING", "propagate": True, }, "django.request": { "handlers": ["file", "console"], "level": "WARNING", # or consider ERROR if this is too noisy "propagate": True, }, # consider adding your own project here if it logs }, ``` - Remove `OVERRIDE_RESOURCE_MODEL_LOCK` if present. 1. Verify ``/apps.py`` no longer has a ``ready()`` method nor the following import: ``` from arches.settings_utils import generate_frontend_configuration ``` 1. Apply migrations: - Within your project, with your Python 3 virtual environment activated run: ``` python manage.py migrate ``` 1. Apply automatic updates: ``` python manage.py updateproject ``` 1. Update your urls.py to include generic error handlers: ``` handler400 = "arches.app.views.main.custom_400" handler403 = "arches.app.views.main.custom_403" handler404 = "arches.app.views.main.custom_404" handler500 = "arches.app.views.main.custom_500" ``` 1. Create draft graphs for your Resource Models: ``` python manage.py graph create_draft_graphs ``` This will publish new versions of each Graph. 1. Update your Graph publications and Resource instances to point to the newly published Graphs: ``` python manage.py graph publish --update -ui ``` 1. Run system checks against your production settings. Note: This command not only performs essential system checks but also generates the frontend configuration file required for your application. It is a crucial step in the deployment process and must be executed to ensure that the environment is correctly configured. ``` python manage.py check --deploy --settings=path.to.production.settings ``` 1. Consider running validation checks: ``` python manage.py validate python manage.py validate --fix [error] ``` 1. Reindex your database: - Within your project with your Python 3 virtual environment activated: ``` python manage.py es reindex_database ``` 1. Update your frontend dependencies: ``` rm -rf node_modules package-lock.json npm install ``` 1. Update your project's `package.json`: - Remove any references to `NODE_OPTIONS` and `NODE_ENV` ```json "scripts": { "build_development": "npm run eslint:check && npm run ts:check && cross-env NODE_OPTIONS=--max-old-space-size=2048 webpack --config ./webpack/webpack.config.dev.js", "build_production": "npm run eslint:check && npm run ts:check && cross-env NODE_OPTIONS=--max-old-space-size=2048 NODE_ENV=production webpack --config ./webpack/webpack.config.prod.js", "build_test": "npm run eslint:check && npm run ts:check && cross-env NODE_OPTIONS=--max-old-space-size=2048 webpack --config ./webpack/webpack.config.dev.js --env test=true", ... "start": "cross-env NODE_OPTIONS=--max-old-space-size=2048 webpack serve --config ./webpack/webpack.config.dev.js", }, ``` ```json "scripts": { "build_development": "npm run eslint:check && npm run ts:check && cross-env webpack --config ./webpack/webpack.config.dev.js", "build_production": "npm run eslint:check && npm run ts:check && cross-env webpack --config ./webpack/webpack.config.prod.js", "build_test": "npm run eslint:check && npm run ts:check && cross-env webpack --config ./webpack/webpack.config.dev.js --env test=true", ... "start": "cross-env webpack serve --config ./webpack/webpack.config.dev.js", }, ``` - Update auto-fixers to apply to staged files only by default: ```json "scripts": { ... "eslint:fix": "eslint **/src --fix", ... "prettier:fix": "prettier {{ project_name }}/src --write", ... }, ``` ```json "scripts": { "eslint:fix": "eslint --fix", "eslint:fix:all": "npm run eslint:fix **/src", ... "prettier:fix": "prettier --write", "prettier:fix:all": "npm run prettier:fix **/src", ... }, ``` 1. Update any imports from `@primevue/themes` to `@primeuix/themes`: ```js // defunct format import { definePreset, palette } from "@primevue/themes"; import Aura from "@primevue/themes/aura"; ``` ```js // updated format import { definePreset, palette } from "@primeuix/themes"; import Aura from "@primeuix/themes/aura"; ``` 1. Update the JavaScript files in your project as follows. (Note that even projects without custom JavaScript may have been templated with a `media/js/reports/default.js`. This file may be removed if it is not intentionally overriding default.js in core Arches. Otherwise, it must be updated.) 1. Update the import and export logic. - Imports should follow the typical ES6 import pattern - Anything `return`-ed from a RequireJS function should be the default export of the file. ```js // defunct format define([ 'knockout', 'viewmodels/report', 'templates/views/report-templates/default.htm', ], function(ko, ReportViewModel, defaultReportTemplate) { return ko.components.register('default-report', { viewModel: function (params) { params.configKeys = []; ReportViewModel.apply(this, [params]); }, template: defaultReportTemplate }); }); ``` ```js // updated format import ko from 'knockout'; import ReportViewModel from 'viewmodels/report'; import defaultReportTemplate from 'templates/views/report-templates/default.htm'; export default ko.components.register('default-report', { viewModel: function (params) { params.configKeys = []; ReportViewModel.apply(this, [params]); }, template: defaultReportTemplate }); ``` 1. If you have created a custom knockout binding, you must bind any declared functions to the object. ```js // defunct format define([ 'knockout', ], function(ko) { ko.bindingHandlers.hover = { init: function(element, valueAccessor) { var value = valueAccessor(); ko.applyBindingsToNode(element, { event: { mouseenter: function() { value(true); }, mouseleave: function() { value(false); } } }); } }; return ko.bindingHandlers.hover; }); ``` ```js // updated format import ko from 'knockout'; ko.bindingHandlers.hover = { init: function(element, valueAccessor) { var value = valueAccessor(); ko.applyBindingsToNode(element, { event: { mouseenter: function() { value(true); }, mouseleave: function() { value(false); } } }); } }; ko.bindingHandlers.hover.init = ko.bindingHandlers.hover.init.bind(ko.bindingHandlers.hover); export default ko.bindingHandlers.hover; ``` 1. If you are importing in the body of your JavaScript components, those imports should be moved to the top. ```js // defunct format define([ 'jquery', 'underscore', 'knockout', 'arches', 'report-templates', 'models/report', 'models/graph', 'viewmodels/alert', 'templates/views/components/resource-report-abstract.htm', 'viewmodels/card', ], function($, _, ko, arches, reportLookup, ReportModel, GraphModel, AlertViewmodel, resourceReportAbstractTemplate) { var ResourceReportAbstract = function(params) { var self = this; var CardViewModel = require('viewmodels/card'); ... ``` ```js // updated format import $ from 'jquery'; import _ from 'underscore'; import ko from 'knockout'; import arches from 'arches'; import reportLookup from 'report-templates'; import ReportModel from 'models/report'; import GraphModel from 'models/graph'; import AlertViewmodel from 'viewmodels/alert'; import resourceReportAbstractTemplate from 'templates/views/components/resource-report-abstract.htm'; import CardViewModel from 'viewmodels/card'; var ResourceReportAbstract = function(params) { var self = this; ... ``` 1. Update the templates in your project: - Any interpolation of `STATIC_URL` should be replaced with `webpack_static`: ```html ``` ```html ``` 1. (optional) Replace `arches.urls` references: - Support for `arches.urls` will be removed in later versions. - Instead of accessing urls via `import arches from "arches";`, you can instead use the `generateArchesURL` utility. ```js // defunct format import arches from "arches"; const url = arches.urls.my_awesome_url( graphId, resourceInstanceId, nodeId, ), fetch(url).then(resp => {...}); ``` ```js // updated format import { generateArchesURL } from "@/arches/utils/generate-arches-url.ts"; const url = generateArchesURL( "my_awesome_url", { graph_id: graphId resource_instance_id: resourceInstanceId, node_id: nodeId } ); fetch(url).then(resp => {...}); ``` - If you update your project to remove any reference to `arches.urls`, you can then safely delete `templates/arches_urls.htm`. 1. If you have been relying on copying client secret values from /o/applications - this release will break that behavior and hash those values by default. Save the value elsewhere before upgrading or delete, and recreate your application(s) to save without hashing. 1. Rebuild your static asset bundle: - If running your project in development: - Option 1: Development Mode (Live Rebuilding) ``` npm start ``` This will build the frontend of the application and then start a webpack development server. The webpack development server will automatically detect changes to bundled assets and rebuild the static asset bundle. This process should take less than 30 seconds. - Option 2: Development Build with Checks ``` npm run build_development ``` This will run ESLint and TypeScript checks, then build the static asset bundle. This process should take less than 30 seconds. - If running your project in production: ``` npm run build_production ``` This will build a minimized, production-ready static asset bundle. This process should take less than 5 minutes. 1. (optional) Collect static files: ``` python manage.py collectstatic ``` This will gather all static files (CSS, JavaScript, Vue, images, etc) and copy them into your project's `STATIC_ROOT` directory. This step does not need to be done if developing locally.