Refactor: Use LF line endings

pull/594/head
Richard 2015-06-30 19:10:54 -06:00
parent 267ff0b93c
commit ac23795778
1 changed files with 95 additions and 95 deletions

View File

@ -1,95 +1,95 @@
Assimp Regression Test Suite Assimp Regression Test Suite
============================ ============================
1) How does it work? 1) How does it work?
--------------------------------------------------------------------------------- ---------------------------------------------------------------------------------
run.py checks all model in the <root>/test/models* folders and compares the result run.py checks all model in the <root>/test/models* folders and compares the result
against a regression database provided with assimp (db.zip). A few failures against a regression database provided with assimp (db.zip). A few failures
are totally fine (see sections 7+). You need to worry if a huge are totally fine (see sections 7+). You need to worry if a huge
majority of all files in a particular format (or post-processing configuration) majority of all files in a particular format (or post-processing configuration)
fails as this might be a sign of a recent regression in assimp's codebase or fails as this might be a sign of a recent regression in assimp's codebase or
gross incompatibility with your system or compiler. gross incompatibility with your system or compiler.
2) What do I need? 2) What do I need?
--------------------------------------------------------------------------------- ---------------------------------------------------------------------------------
- You need Python installed (2.7+, 3.x). On Windows, run the scripts using "py". - You need Python installed (2.7+, 3.x). On Windows, run the scripts using "py".
- You need to build the assimp command line tool (ASSIMP_BUILD_ASSIMP_TOOLS - You need to build the assimp command line tool (ASSIMP_BUILD_ASSIMP_TOOLS
CMake build flag). Both run.py and gen_db.py take the full path to the binary CMake build flag). Both run.py and gen_db.py take the full path to the binary
as first command line parameter. as first command line parameter.
3) How to add more test files? 3) How to add more test files?
--------------------------------------------------------------------------------- ---------------------------------------------------------------------------------
Use the following procedure: Use the following procedure:
- Verify the correctness of your assimp build - run the regression suite. - Verify the correctness of your assimp build - run the regression suite.
DO NOT continue if more tests fail than usual. DO NOT continue if more tests fail than usual.
- Add your additional test files to <root>/test/models/<fileformat>, where - Add your additional test files to <root>/test/models/<fileformat>, where
<fileformat> is the file type (typically the file extension). <fileformat> is the file type (typically the file extension).
- If you test file does not meet the BSD license requirements, add it to - If you test file does not meet the BSD license requirements, add it to
<root>/test/models-nonbsd/<fileformat> so people know to be careful with it. <root>/test/models-nonbsd/<fileformat> so people know to be careful with it.
- Rebuild the regression database: - Rebuild the regression database:
"gen_db.py <binary> -ixyz" where .xyz is the file extension of the new file. "gen_db.py <binary> -ixyz" where .xyz is the file extension of the new file.
- Run the regression suite again. There should be no new failures and the new - Run the regression suite again. There should be no new failures and the new
file should not be among the failures. file should not be among the failures.
- Include the db.zip file with your Pull Request. Travis CI enforces a passing - Include the db.zip file with your Pull Request. Travis CI enforces a passing
regression suite (with offenders whitelisted as a last resort). regression suite (with offenders whitelisted as a last resort).
4) I made a change/fix/patch to a loader, how to update the database? 4) I made a change/fix/patch to a loader, how to update the database?
--------------------------------------------------------------------------------- ---------------------------------------------------------------------------------
- Rebuild the regression database using "gen_db.py <binary> -ixyz" - Rebuild the regression database using "gen_db.py <binary> -ixyz"
where .xyz is the file extension for which the loader was patched. where .xyz is the file extension for which the loader was patched.
- Run the regression suite again. There should be no new failures and the new - Run the regression suite again. There should be no new failures and the new
file should not be among the failures. file should not be among the failures.
- Include the db.zip file with your Pull Request. Travis CI enforces a passing - Include the db.zip file with your Pull Request. Travis CI enforces a passing
regression suite (with offenders whitelisted as a last resort). regression suite (with offenders whitelisted as a last resort).
5) How to add my whole model repository to the database? 5) How to add my whole model repository to the database?
--------------------------------------------------------------------------------- ---------------------------------------------------------------------------------
Edit the reg_settings.py file and add the path to your repository to Edit the reg_settings.py file and add the path to your repository to
<<model_directories>>. Then, rebuild the database. <<model_directories>>. Then, rebuild the database.
6) So what is actually tested? 6) So what is actually tested?
--------------------------------------------------------------------------------- ---------------------------------------------------------------------------------
The regression database includes mini dumps of the aiScene data structure, i.e. The regression database includes mini dumps of the aiScene data structure, i.e.
the scene hierarchy plus the sizes of all data arrays MUST match. Floating-point the scene hierarchy plus the sizes of all data arrays MUST match. Floating-point
data buffers, such as vertex positions are handled less strictly: min, max and data buffers, such as vertex positions are handled less strictly: min, max and
average values are stored with low precision. This takes hardware- or average values are stored with low precision. This takes hardware- or
compiler-specific differences in floating-point computations into account. compiler-specific differences in floating-point computations into account.
Generally, almost all significant regressions will be detected while the Generally, almost all significant regressions will be detected while the
number of false positives is relatively low. number of false positives is relatively low.
7) The test suite fails, what do do? 7) The test suite fails, what do do?
--------------------------------------------------------------------------------- ---------------------------------------------------------------------------------
Get back to <root>/test/results and look at regression_suite_failures.txt. Get back to <root>/test/results and look at regression_suite_failures.txt.
It contains a list of all files which failed the test. Failing dumps are copied to It contains a list of all files which failed the test. Failing dumps are copied to
root>/test/results/tmp. Both an EXPECTED and an ACTUAL file is produced per test. root>/test/results/tmp. Both an EXPECTED and an ACTUAL file is produced per test.
The output of "assimp cmpdump" is written to regressions_suite_output.txt. Grep The output of "assimp cmpdump" is written to regressions_suite_output.txt. Grep
for the file name in question and locate the log for the failed comparison. It for the file name in question and locate the log for the failed comparison. It
contains a full trace of which scene elements have been compared before, which contains a full trace of which scene elements have been compared before, which
makes it reasonably easy to locate the offending field. makes it reasonably easy to locate the offending field.
8) fp:fast vs fp:precise fails the test suite (same for gcc equivalents) 8) fp:fast vs fp:precise fails the test suite (same for gcc equivalents)
--------------------------------------------------------------------------------- ---------------------------------------------------------------------------------
As mentioned above, floating-point inaccuracies between differently optimized As mentioned above, floating-point inaccuracies between differently optimized
builds are not considered regressions and all float comparisons done by the test builds are not considered regressions and all float comparisons done by the test
suite involve an epsilon to accomodate. However compiler settings that allow suite involve an epsilon to accomodate. However compiler settings that allow
compilers to perform non-IEEE754 compliant optimizations can cause arbitrary compilers to perform non-IEEE754 compliant optimizations can cause arbitrary
failures in the test suite. Even if the compiler is configured to be IEE754 failures in the test suite. Even if the compiler is configured to be IEE754
comformant, there is lots of code in assimp that leaves the compiler a choice comformant, there is lots of code in assimp that leaves the compiler a choice
and different compilers make different choices (for example the precision of and different compilers make different choices (for example the precision of
float intermediaries is implementation-specified). float intermediaries is implementation-specified).