Commit graph

97 commits

Author SHA1 Message Date
Dirk Koritnik
4a06d0d46a
Extend import to get copyrights (#53) 2024-04-23 11:14:43 +02:00
ab75902a95
Fix broken assignment of features and categories (#41)
We migrated the part of the import to use DataHandler.
We didn't invest too much time as budgets are low.
Still the bugs are covered with tests and fixed.

Relates: #10782
2023-11-06 08:44:03 +01:00
27ee70d0cf
Add event to modify categories during destination.one import (#34)
A new PSR-14 event is added that allows to modify the categories to be
assigned to an event.
The event itself (including already existing categories) as well as the
list of categories to be used after import are available.
It is possible to change the categories to be assigned, e.g. keep some
of the existing categories.

That way it is possible for installations to add custom categories to
events.

Relates: #10623
2023-08-14 12:09:28 +02:00
5af1d08148
Add source while importing from Destination Data One (#37)
Destination Data One provides an source info explaining where the data
originally came from.
This is now added to event records and exposed in TYPO3 backend.
That allows editors to check the source and contact corresponding
sources in order to fix broken data.

Relates: #10630
2023-08-10 15:02:05 +02:00
3d0d5d8645
Add PSR-14 Event to modify event objects prior persistence (#35)
A new PSR-14 event is added that allows to modify the event right before
it is persisted within Destination Data One import.

This for example allows to alter dates, prices, etc.

Improve handling of missing end time in dates.
2023-08-10 14:20:37 +02:00
00946af6ad
Properly handle latitude and longitude during Destination One import (#32)
They sometimes use a different separator.
The code is adjusted to always use same separator and precision.

That will prevent the same location from showing up multiple times due
to different latitude and longitude values.
2023-07-06 12:39:47 +02:00
9f0eaac154
Improve CGL 2023-07-04 10:21:49 +02:00
0f7323eac3
Fix removal of still used files (#27)
Simplify SQL queries and move logic to PHP.
2023-06-20 11:56:59 +02:00
c56a10b748
Improve meta data of imported files (#24)
* Set ``alternative`` to the ``description`` if available.

* Set ``creator_tool`` to the "destination.one".

* Set ``source`` to the URL of the fetched image.
2023-06-19 10:22:54 +02:00
bae680025e
Flush corresponding page caches on import (#25)
The pages now receive proper cache tags.
The import now properly clears those cache tags.
That way all corresponding pages will show updated content after import
finished.

We need one test that executes frontend requests and import command.
The separation is therefore removed and tests are streamlined to have a
single parent providing all necessary information and setup.
2023-06-19 10:13:26 +02:00
7b3bbf6d9d
Do not download already existing files during import (#26)
The usual use cases right now would involve a new file with a new URL.
That way we don't need to download existing files again.
That reduced network traffic, execution time and resource usage.
2023-06-19 10:13:17 +02:00
d9fef5367a
Reduce broken time frame due to missing slugs (#23)
Dates are vanished and re created during import.
Slugs for dates were generate after whole import was finished.
This is now changed to generate slugs for dates after saving each single
event.
2023-06-15 09:55:15 +02:00
a756998f1f
Add proper image handling during import (#22)
Import now properly handles images.
It only added missing images beforehand but now will properly:
- Remove
- Resort
- Add
- Update

Relates: #10521
2023-06-12 11:07:52 +02:00
d4a11436c6
Streamline CGL (#21)
Use php-cs-fixer as within other extensions.
Use same configuration.
Apply configuration
2023-06-07 08:56:42 +02:00
9c8e1aa982
Do not delete files if they are still in use (#12)
The clean up task had an issue to clean up to many files.
It didn't check if a file that could be deleted
because one event got deleted, was still in use by another event.

Relates: #10499
2023-05-15 15:35:48 +02:00
a49ba7c4d2
Fix 404 for dates after re import (#7) 2023-04-03 11:34:56 +02:00
Dirk Koritnik
f3b3a70948
Remove rte config fram tca, fix text cleanup from linebreaks (#3) 2023-03-17 12:58:06 +01:00
Dirk Koritnik
b99f19b79d Allow png images in file import from destination one 2023-01-24 12:01:27 +00:00
f68da60842 Provide configurable repeatUntil fallback 2023-01-05 09:01:13 +00:00
6570ec5ca9 Properly handle guzzle exception for fetching images 2022-12-21 12:16:20 +00:00
fcbda4c093 Add default routing 2022-11-23 09:59:55 +00:00
6412c31be5 Do not delete files unrelated to events 2022-11-22 14:59:06 +00:00
e1674b1472 Fix broken timezone during import 2022-08-29 12:44:36 +00:00
239bfe51fc Remove slugs from locations 2022-08-03 09:24:01 +00:00
5652ea6442 Fix broken import without features 2022-08-03 08:57:52 +02:00
65e653f6ec Add location as separate record 2022-08-02 13:56:18 +00:00
0a56a5d482 Integrate destination data features
Import alongside of categories.
Use TYPO3 sys_category again.
2022-07-13 15:59:42 +02:00
6e6e914e36 Ensure first date of recurring daily event is created 2022-07-13 13:58:05 +00:00
b0dae68550 Fix broken timezone and end time after import of recurring events 2022-07-07 12:10:49 +00:00
6b6ae5b20c Do not break tests once a certain date is passed
The code used native date functions, this made tests not robust.
We switch to TYPO3 API to fetch the current date and time.
Test is adjusted to define the date and time to verify code works as
expected.
2022-07-07 09:10:14 +02:00
3d98475a6c Fix none working import of ticket links from destination data
The existing code didn't work with the provided data.
We added a new test case to cover the situation.
We adjusted the existing code to check on the actual rel value instead
of the keys.
2022-07-07 09:03:29 +02:00
96f07fd501 Ensure title of sys_file_metadata is not exceeded
The columns is limited by length.
We ensure we do crop incoming file names before adding them to the
database.
We would run into exception otherwise.
2022-06-28 11:39:12 +02:00
fc8c2b3046 Do not enter endless loop while date creation
Entering the continue part would end in an endless loop.
We never add anything to $i and therefore would end up here.

Existing tests ensure we don't break something.
The new test ensures we do not hit endless loop anymore.
2022-06-27 16:07:36 +02:00
c506720689 Respect that interval is now always set within timeIntervals
The property will always be set in the future.
We no longer can use it as an indicator.

We switch to freq as new indicator.
We also ensure that start has to be given for a single date.

Relates: #9892
2022-04-25 07:47:47 +02:00
4ef67b9bae Refactor date creation during import
Split into own class to encapsulate the logic.
2022-04-21 09:30:23 +02:00
3d6bf0ac8a Add import configuration record
It is now possible to create "Import" records in TYPO3 system.
Those records can then be imported via two new commands.
There is no need to configure everything within the command itself.

That allows:
    * To allow editors to maintain import configuration.
    * Have proper labels, description, csh and UI to create import.
      It no longer is necessary to copy UIDs,
      instead proper wizards from TCA are available to select pages.

Relates: #9649
2022-01-27 15:50:36 +01:00
4bc4f34525 Add support for search query in DO import command
Relates: #9648
2022-01-27 07:57:05 +01:00
b5d095659a Use new import class and move url building and fetching 2022-01-26 15:15:15 +01:00
c6ca707b1c Fix warnings in higher PHP versions ending up in TYPO3 log 2021-12-20 10:20:47 +01:00
06d111e132 Do not trigger warning if no category was configured
The import already worked without category, but triggered a warning.
Tests were extended to always work with category, except in new test
case.
All tests were extended to check for empty logs in the end.

The import now checks whether category should be imported and ignores
further processing if not configured.

Relates: #9542
2021-12-20 10:15:26 +01:00
eecbb1feb4 Remove useless file
This got introduced somewhere during refactoring but was left empty
anyway.
2021-12-20 10:15:09 +01:00
ab9c3e0c4e Fix broken remove past data
Past data is now removed again.
A test ensures the functionality.
Data is no longer marked as deleted but is actually deleted.

Relates: #9543
2021-12-20 09:45:25 +01:00
6aa4681cb9 Delete files which were referenced by events objects
Adjust existing code base to not use hardcoded file path.
Instead use info from database and check all files which have references
to event objects.

Also add test to cover feature.
2021-12-16 14:07:32 +01:00
15dec49445 Remove usage of uploads folder, use transient API instead
Relates: #9533
2021-12-16 10:00:51 +01:00
de82e395e6 Fix Coding Guideline check on CI
The current setup actually fixed and didn't check or fail.
That's changed and code is fixed.
2021-12-16 09:13:15 +01:00
0e5cd38176 Add first basic functional test for import
The test ensures import runs with a small example set of events.

We needed to alter fetching of images, in order to properly mock
responses.
2021-12-16 09:08:16 +01:00
73161fa24e Make region optional during import 2021-12-14 17:31:56 +01:00
29c31375a5 Remove PHPStan issues
Most were fixed, some were ignored on top level.
2021-12-14 17:31:46 +01:00
0d0c64f419 Fix PHPStan issues 2021-12-13 09:27:28 +01:00
b430993655 Remove execution rights from php file 2021-12-13 08:51:32 +01:00