p4 journaldbchecksums
Write journal notes with table checksums.
Syntax
p4 [g-opts] journaldbchecksums [-t tableincludelist | -T tableexcludelist] [-l level]
p4 [g-opts] journaldbchecksums -u filename -t tablename [-v version] [-z]
p4 [g-opts] journaldbchecksums -s -t tablename [-b blocksize] [-v version]
p4 [g-opts] journaldbchecksums -c changelist
Description
The p4 journaldbchecksums command provides a set of
tools for ensuring data integrity across a distributed or replicated
installation.
Helix Core Server automatically performs an integrity check when you use the
p4 admin checkpoint or p4 admin
journal commands, or when you use p4
journaldbchecksums to manually perform an integrity
check.
To use this command with structured logging (see p4 logparse), at least one structured log must be capturing events of type
integrity.
When an integrity check is performed, the
Server writes records to the journal that contains the checksums of the
specified tables. If no tables are specified, the
Server writes records to the journal that contains the checksums for all tables.
Replica servers, upon receiving these records, compare these checksums
with those computed against their own database tables, as they would with
p4 dbstat. Results of
the comparisons are written in the replica’s log.
You can control which tables are checked, by:
|
including individual tables with |
||||||||
|
excluding individual tables with |
||||||||
|
using one of three levels of verification:
Verification levels are controlled by the |
When checking individual changelists and individual tables, the
rpl.checksum.change and the rpl.checksum.table
configurables control when events are written to the log.
See also Verifying replica integrity in the Helix Core Server Administrator Guide.
Options
|
|
When scanning tables, scan blocksize records per block. The default is 5,000. For each block, the server computes a block checksum and writes it as a journal note. Replica servers automatically verify these blocks when processing these notes. This option can be used with large tables on a production system as the table is unlocked between each block. Inspecting the results of the block verifications will reveal the location of damage that affects only part of a database table. See the example of Database Table Block Checksums. |
|
|
Compute a checksum for an individual submitted changelist. The checksum is written as a journal note, and replica servers automatically verify the checksum of the change when they process these notes. See the example of Changelist Checksums. |
|
|
Specify a level for checksumming; each level corresponds to a
larger set of tables. These levels correspond to the levels used
by the |
|
|
Scan the specified database table. See the example of Database Table Unload. |
|
|
Specify the table(s) for which to compute checksums. To specify
multiple tables, double-quote the list and separate the table
names with spaces. The table names must start with
" |
|
|
Compute checksums for all tables except those listed. |
|
|
Unload to the specified filename the specified tablename. This command also writes a journal note that documents this action and instructs replica servers to unload the same table to the same file when they process this journal note. Prerequisite: The |
|
|
When unloading or scanning tables, specify the server version number to use. If no server version number is specified, the current server version is used. |
|
|
Compress the file when unloading a table. |
|
|
See Global options. |
Usage notes
| Can File Arguments Use Revision Specifier? | Can File Arguments Use Revision Range? | Minimal Access Level Required |
|---|---|---|
|
N/A |
N/A |
available to an operator user |
For more about administering Helix Core Server in distributed or replicated environments, see Helix Core Server Administrator Guide.
Examples
Database Table Checksums
p4 journaldbchecksums [-t tableincludelist | -T tableexcludelist] [-l N]
causes the server to write journal notes containing table checksum information:
p4 journaldbchecksums -t db.rev @nx@ 12 1487712216 @41@ 9 -933920831 0 4 0 @db.rev@ @@ @@ @@ @@
Edge/Replica servers automatically verify the table checksums when processing these notes, writing the results to the server log and optionally an integrity structured log if configured:
Table db.rev checksums match. 2017/02/21 13:23:36 version 9: expected 0xC8557FC1, actual 0xC8557FC1 p4 logparse -m1 -F f_table=db.rev -T 'f_date f_results' integrity.csv ... f_date 2017/02/21 13:23:36 219149298 ... f_results match
The results of table checksum comparison will result in one three possible results:
- match
- DIFFER
- empty
Table db.have checksums DIFFER. 2017/02/21 13:08:38 version 3: expected 0x3BB210EE, actual 0xB1BF3E83 p4 logparse -F f_results=DIFFER -T 'f_date f_table' integrity.csv ... f_date 2017/02/21 13:08:38 203821071 ... f_table db.have
Table db.ldap checksums empty. 2017/02/24 11:33:54 version 0: expected 0x0, actual 0x0. p4 logparse -F f_results=empty -T f_table integrity.csv ... f_table db.ldap
The table checksums might be reported as DIFFER if the database structure diverged as the result of:
|
Software upgrades: |
Some upgrades are performed against a database table when data in a table is accessed. |
|
Replaying a checkpoint or journal:
|
When administrators replay journal data or journal patches using p4d -jr, the transactions replayed into the database are not journaled. This might generate differing checksums. When replaying journal data in a distributed environment, always use p4d -s -jr so the replayed transactions are journaled, thus enabling downstream edge/replica servers to replay them. Be aware that p4d -jr run against a replica server only updates the replica's database files and therefore might generate differing checksums. |
|
Journal filtering: |
When filtering is active in your replication process, not all journal checksums are expected to match. |
To remedy unexpected checksum differences, restore the edge/replica server database from a new checkpoint of the commit/master server.
Changelist Checksums
p4 journaldbchecksums -c change
causes the server to compute a checksum of an individual submitted changelist. This checksum is written as a journal note:
p4 journaldbchecksums -c 12073 @nx@ 15 1487961638 @41@ 12073 1 0 0 0 @46B19358420B468668781A002BA0AC15@ @@ @@ @@ @@
Replica servers automatically verify the checksum of the change when processing these notes and write the results to the integrity structured log:
p4 logparse -F f_change=12073 -T f_results integrity.csv
... f_results match
Server behavior is dependent on the rpl.checksum.change configurable.
Database Table Block Checksums
p4 journaldbchecksums -s -t tablename [ -b blocksize ][-v N]
causes the server to scan the specified database table. The table is scanned in blocks. The number of records in a block is specified by the -b flag, which defaults to 5,000. For each block, the server computes a block checksum and writes it as a journal note:
p4 journaldbchecksums -s -t db.have @nx@ 17 1487964567 @41@ 3 1 313 0 0 @db.have@ @@@//Talkhouse/build/jar/Talkhouse.jar@@ @ @@@//Jam/MAIN/src/glob.c@@ @ @2BCDA450287C03DE3433AEB6278EA4AA@ @@
Replica servers automatically verify these blocks when processing these notes and write output to the integrity structured log if configured:
p4 logparse -F 'f_table=db.have' -T 'f_results f_checkSum f_checkSum2' integrity.csv
... f_checkSum 2BCDA450287C03DE3433AEB6278EA4AA
... f_checkSum2 D41D8CD98F00B204E9800998ECF8427E
... f_results failed
This command can be used with large tables on a production system because the table is unlocked between each block. Inspecting the results of the block verifications reveals the location of any damage, which affects only part of a database table.
Database Table Unload
p4 journaldbchecksums -u filename -t tablename [-v N] [-z]
causes the server to unload the specified database table to the specified file. The command also writes a journal note describing this action:
p4 journaldbchecksums -u working.txt -t db.working @nx@ 16 1487964861 @41@ 10 0 0 0 0 @db.working@ @working.txt@ @@ @@ @@
Replica servers automatically unload the same table to the same file when processing these notes. If only a file name is specified with -u, as in the example above, the unload files are created in the P4ROOT directory of both servers. Any relative path specified with -u is relative to the P4ROOT directory. Absolute paths to the unload file can also be used. Ensure any referenced directory paths exists on both master and replica prior to running the unload.
For a time-consistent comparison of the contents of the table, unload the tables in this way. This command is recommended only for tables that are small. The -z flag specifies that the file should be compressed.
