Asset Deletion 3 - Monitoring
While Asset Deletion 3 is executing the deletion process, it writes status information into the log. Use this information to monitor the deletion process. You can also check the database for some more information.
Context
censhare provides log information to monitor the deletion process. Both, the individual and the automatic server actions write information into the log.
- A report checks which constraints exist for the different versions of an asset.
Prerequisites
Understanding of the deletion process and the server actions of Asset Deletion 3
Access to the server log files
Knowledge of how to work on the command shell of a server
Knowledge of how to work with grep
Access to the database
Database administration tool
Knowledge of SQL queries
Introduction
When censhare executes Asset Deletion 3, it writes information about each stage of the process into the server log file.
Within each stage of the deletion process, censhare writes the key data of the processing step into the log file.
censhare provides three server actions for Asset Deletion 3:
Delete Asset Versions (automatic)
Delete Asset Versions (individual)
Asset Deletion 3 Report
Each of these three server actions writes its own entries into the server log. You can identify them by keywords:
Keyword | Remark |
AssetDeletion3Config | Use this keyword to find all entries related to the configuration check. |
AssetDeletion3Handler | Use this keyword to find all entries related to any part of the asset deletion process. |
startAssetDeletion3 | Use this keyword to find all entries that are related to the automatic server action. |
startIndividualAssetDeletion3 | Use this keyword to find all entries that are related to the individual server action. |
ad3DeletionReport | Use this keyword to find all entries that are related to the Asset Deletion 3 Report. |
Automatic server action: Log messages
Note: All log entries are only examples. The entries in your system can look different.
Note: For the two first stages of the deletion process use the batch concept.
Configuration check
Before executing the automatic server action, censhare checks if any automatic server action of Asset Deletion 1 is activated. It then writes a warning into the server log. The automatic server action of Asset Deletion 3 is not executed. This is to prevent the parallel execution of Asset Deletion 1 and 3.
Mark asset versions
Automatic server action: Start of the Mark asset versions stage
.. AssetDeletion3Config.Check.configCheck: CommandExecutor:
master-postgres.20190116.150608.063[USER]: AD.automatic task is enabled
(asset_deletion.storage-version-deletion,
relPath: file:modules/asset_deletion/storage-verson-deletion_de.xml),
please disabled it and make sure the rest are as well on Java Admin
in order to run AD3
The 10 minutes time-out for Mark asset versions is defined in the censhare Admin Client..
USER is the user "system" for the automatic server action. The log entry lists all server actions that are activated for Asset Deletion 1. relPath shows the relative path of the configuration file of the server action in the censhare-Custom directory. For more information, see Asset Deletion 3: Configure server actions for deletion.
Start
Start of the automatic server action:
... AssetDeletion3Handler.startAssetDeletion3: CommandExecutor:
master-postgres.20190116.150608.063[USER]: Asset Deletion 3 has started
USER is the user "system" for the automatic server action.
Mark asset versions
Automatic server action: Start of the Mark asset versions stage
... AssetDeletion3Handler.startAssetDeletion3: CommandExecutor:
master-postgres.20190116.115621.446[USER]: Marking asset has started with
a time-out of 10 minutes
The 10 minutes time-out for Mark asset versions is defined in the censhare Admin Client.
Starting to analyze a new batch:
... AssetDeletion3Handler.startAssetDeletion3: CommandExecutor:
master-postgres.20190116.115621.446[USER]: Amount of candidates
from the batch: 32
This batch has 32 assets with candidates for asset versions that are to be marked for deletion. Constraints on asset and asset version level are applied.
... AssetDeletion3Handler.startAssetDeletion3: CommandExecutor:
master-postgres.20190116.115621.446[USER]: Amount of candidate
to be marked (after the policies have been applied) : 20
This entry shows how many asset versions of the asset are left after the Keep versions parameters are applied. 20 is the final number of candidates that are marked for deletion.
For each asset version that is marked for deletion, a log file entry is created:
... AssetDeletion3Handler.startAssetDeletion3: AssetManagementService:
master-postgres.20190116.115621.446[USER]: Deletion [marked for deletion]
of asset: AOUA96RLBV1 .jpg[13892-v2-c2-tcn12-ccn22]
The asset version 2 (13892-v2-c2-tcn12-ccn22) of the asset AOUA96RLBV1.jpg with the ID 13892 (13892-v2-c2-tcn12-ccn22) was marked for deletion.
After marking the asset versions for deletion in the actual batch, censhare writes the total number of marked asset versions into the server log:
... AssetDeletion3Handler.startAssetDeletion3: CommandExecutor:
master-postgres.20190116.115621.446[USER]: Amount of marked assets
for deletion within the batch: 20
In this batch, 20 asset versions were marked for deletion. If this number is lower than the number of candidates, there have been errors when committing changes in the database. For example, the update database action violates a database constraint.
When all batches of the first stage are processed, a log entry is created:
... AssetDeletion3Handler.startAssetDeletion3: CommandExecutor:
master-postgres.20190116.115621.446[USER]: Marking asset action
has finished with a total of 258 marked assets
Delete asset versions
Automatic server action: Start of the Delete asset versions stage:
... AssetDeletion3Handler.startAssetDeletion3: CommandExecutor:
master-postgres.20190116.115621.446[USER]: Delete asset has started with
a time-out of 30 minutes
The 30 minutes time-out is defined in the censhare Admin Client.
censhare checks for each asset version in a batch if the Keep for hours rule still applies:
... AssetDeletion3Handler.startAssetDeletion3: CommandExecutor:
master-postgres.20190116.150608.063[USER]: Asset to be deleted after
policy (keep for hours) has been applied: 32
For each asset version that is deleted, a log entry is created:
... AssetDeletion3Handler.startAssetDeletion3: AssetManagementService:
master-postgres.20190116.150608.063[USER]: Deletion [physical deletion] of
asset: navigation.jpeg[13881-v2-c2-tcn18-ccn18]
Version 2 (v2) of the navigation.jpeg asset (ID 13881) is deleted.
After deleting the asset versions in a batch, censhare creates a log entry with the total number of deleted versions:
... AssetDeletion3Handler.startAssetDeletion3: CommandExecutor:
master-postgres.20190116.150608.063[USER]: Amount of
assets physically deleted : 32
In this batch, 32 asset versions have been deleted. If this number is lower than the number of assets to be deleted after the policy check, there have been errors when executing the database deletion commands.
When the Delete asset versions stage is finished, the total number of all deleted asset versions is entered:
... AssetDeletion3Handler.startAssetDeletion3: CommandExecutor:
master-postgres.20190116.150608.063[USER]: Deleting asset (doDeletion)
action has finished with a total of 70 deleted assets
Delete storage items
Automatic server action: Start of the Delete storage items stage:
... AssetDeletion3Handler.startAssetDeletion3: CommandExecutor: SERVER_NAME: Storage Deletion batch timed out after 120 minutes
When the Deletion storage item stage is finished, censhare writes the number of deleted asset files in the log:
Deletion storage item stage has finished:
... AssetDeletion3Handler.startAssetDeletion3: CommandExecutor: SERVER_NAME: Storage item deletion has finished; 84798 unused files deletedd
Finished
The automatic server action is finished:
... AssetDeletion3Handler.startAssetDeletion3: CommandExecutor:
master-postgres.20190116.150608.063[USER]: Asset Deletion 3 has finished
Individual server action: Log messages
The log messages for the individual server action are similar to the log messages of the automatic server action. The manual server action is logged with startIndividualAssetDeletion3 instead of startAssetDeletion3.
The user who executes the individual server action is logged as well:
... AssetDeletion3Handler.startIndividualAssetDeletion3: CommandExecutor:
SERVER_NAME.20190123.174119.754[USER]: Individual asset deletion has
started with a time-out of 10 minutes
USER is the login name of the user who started the individual server action.
Deletion report: Log messages
Start of the report:
... AssetDeletion3Handler.ad3DeletionReport: CommandExecutor:
SERVER_NAME.20190222.154355.005[USER]: Asset Deletion 3 Report has started
USER is the login name of the user who executes the report server action.
End of the report:
... AssetDeletion3Handler.ad3DeletionReport: CommandExecutor:
SERVER_NAME.20190222.154355.005[USER]: Asset Deletion 3 Report has finished
Analyze log files via grep
A simple way to analyze the server log is grep. Use the following commands to find entries in the server log that are related to the asset deletion process.
Find all entries related to Asset Deletion 3:
grep AssetDeletion3Handler server-0.*
Find all entries related to automatic server action:
grep startAssetDeletion3 server-0.*
Find all entries related to individual server action:
grep startIndividualAssetDeletion3 server-0.*
Search for specific results of a server action like a list of all asset versions marked for deletion:
grep startAssetDeletion3 server-0.* | grep "marked for deletion"
Search for the result of the Marked for deletion stage:
grep "Marking asset action has finished with a total of" ~/work/logs/server-0.*.log
Search for the result of the Delete asset versions stage:
grep "Deleting asset.*action.*finished with a total of" ~/work/logs/server-0.*.log
Search for the result of the Delete storage items stage:
grep -i "storage.*deletion.*batch" server-0.*.log
grep -i "storage.*deletion.*finished" server-0.*.log
Database
Check for Marked for deletion
Execute the following SQL command (PostgrSQL):
select count(*) from asset where deletion = 1;
The command counts all entries in the asset table where the deletion attribute of the asset is set to "1". This is the value for Marked for deletion.
Use the value to check if the last run of the automatic server action has physically removed all asset versions that are marked for deletion. If not, you can increase the time frame for the Mark asset versions stage.
Check for file deletion
Execute the following SQL command (PostgreSQL):
select count(*) from storage_deletion_queue;
This command counts all entries in the storage_deletion_queue table. Each entry represents an asset file.
If there are asset files left in the queue, you can increase the time for the Delete storage items stage of the deletion process.
Note: There can be entries in the storage_deletion_queue for which the corresponding asset file cannot be deleted. For example, it is still used by another asset.
Result
You understand which log entries are created for the different parts of the asset deletion process. You can use grep to find specific entries of the deletion process in the log. You can use SQL to get some basic information about the results after an asset deletion process.