Add first version of regression suite (python parts only). Work in progress.
git-svn-id: https://assimp.svn.sourceforge.net/svnroot/assimp/trunk@597 67173fc5-114c-0410-ac8e-9d2fd5bffc1fpull/1/head
parent
11aa5c339d
commit
30ca88e782
|
@ -0,0 +1,17 @@
|
||||||
|
<?xml version="1.0" encoding="UTF-8"?>
|
||||||
|
<projectDescription>
|
||||||
|
<name>assimp-regression</name>
|
||||||
|
<comment></comment>
|
||||||
|
<projects>
|
||||||
|
</projects>
|
||||||
|
<buildSpec>
|
||||||
|
<buildCommand>
|
||||||
|
<name>org.python.pydev.PyDevBuilder</name>
|
||||||
|
<arguments>
|
||||||
|
</arguments>
|
||||||
|
</buildCommand>
|
||||||
|
</buildSpec>
|
||||||
|
<natures>
|
||||||
|
<nature>org.python.pydev.pythonNature</nature>
|
||||||
|
</natures>
|
||||||
|
</projectDescription>
|
|
@ -0,0 +1,7 @@
|
||||||
|
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
|
||||||
|
<?eclipse-pydev version="1.0"?>
|
||||||
|
|
||||||
|
<pydev_project>
|
||||||
|
<pydev_property name="org.python.pydev.PYTHON_PROJECT_VERSION">python 3.0</pydev_property>
|
||||||
|
<pydev_property name="org.python.pydev.PYTHON_PROJECT_INTERPRETER">Default</pydev_property>
|
||||||
|
</pydev_project>
|
|
@ -0,0 +1,97 @@
|
||||||
|
Assimp Regression Test Suite
|
||||||
|
============================
|
||||||
|
|
||||||
|
1) How does it work?
|
||||||
|
---------------------------------------------------------------------------------
|
||||||
|
run.py checks all model in the <root>/test/models folder and compares the result
|
||||||
|
against a regression database provided by us (db.zip). If the test passes
|
||||||
|
successfully, Assimp definitely WORKS perfectly on your system. A few failures
|
||||||
|
are totally fine as well (see sections 7+). You need to worry if a huge
|
||||||
|
majority of all files in a particular format or post-processing configuration
|
||||||
|
fails - this might be a sign of a recent regression in assimp's codebase.
|
||||||
|
|
||||||
|
2) What do I need?
|
||||||
|
---------------------------------------------------------------------------------
|
||||||
|
- you need Python installed - 3.x !!
|
||||||
|
- you need to build tools/assimp_cmd as described in the INSTALL file (
|
||||||
|
make && make install on unixes,release-dll target with msvc).
|
||||||
|
|
||||||
|
3) How to add more test files?
|
||||||
|
---------------------------------------------------------------------------------
|
||||||
|
Use the following procedure:
|
||||||
|
- verify the correctness of your assimp build - run the regression suite.
|
||||||
|
DO NOT continue if one or more tests fail.
|
||||||
|
- add your additional test files to <root>/test/models/<fileformat>, where
|
||||||
|
<fileformat> is the file type (typically the file extension)
|
||||||
|
- rebuild the regression database using gen_db.py
|
||||||
|
- run the regression suite again - all tests should pass, including
|
||||||
|
those for the new files.
|
||||||
|
|
||||||
|
- contributors: commit the db.zip plus your additional test files to
|
||||||
|
the SVN repository.
|
||||||
|
|
||||||
|
4) I made a change/fix/.. to a loader, how to update the database?
|
||||||
|
---------------------------------------------------------------------------------
|
||||||
|
- rebuild the regression database using gen_db.py
|
||||||
|
- run the suite - all tests should pass now. If they do not, don't continue
|
||||||
|
|
||||||
|
- contributors: commit the db.zip to the SVN repository
|
||||||
|
|
||||||
|
5) How to add my whole model repository to the suite?
|
||||||
|
---------------------------------------------------------------------------------
|
||||||
|
|
||||||
|
Edit the reg_settings.py file and add the path to your repository to
|
||||||
|
<<model_directories>>. Then, rebuild the suite.
|
||||||
|
|
||||||
|
6) So what is actually verified?
|
||||||
|
---------------------------------------------------------------------------------
|
||||||
|
The regression database includes mini dumps of the aiScene data structure -
|
||||||
|
the scene hierarchy plus the sizes of all data arrays MUST match. Floating-point
|
||||||
|
data buffers, such as vertex positions, are handled less strictly: min,max and
|
||||||
|
average values are stored with lower precision. This takes hardware- or
|
||||||
|
compiler-specific differences in floating-point computations into account.
|
||||||
|
Generally, almost all significant regressions will be detected while the
|
||||||
|
number of false positives is relatively low.
|
||||||
|
|
||||||
|
7) The test suite fails, what do do?
|
||||||
|
---------------------------------------------------------------------------------
|
||||||
|
Get back to ../results and check out regression_suite_failures.txt
|
||||||
|
It contains a list of all files which failed the test ... they're copied to
|
||||||
|
../results/tmp. Both an EXPECTED and an ACTUAL file is produced per test.
|
||||||
|
The output of `assimp cmpdump` is written to regressions_suite_output.txt.
|
||||||
|
To quickly find all all reports pertaining to tests which failed, I'd
|
||||||
|
recommend grepping for 'but' because its a constituent of all error messages
|
||||||
|
produced by assimp_cmd :) Error reports contain detailed information
|
||||||
|
regarding the point of failure inside the data structure, the values of
|
||||||
|
the two corresponding fields that were found to be different from EXPECTED
|
||||||
|
and ACTUAL, respectively, ... this should offer you enough information to start
|
||||||
|
debugging.
|
||||||
|
|
||||||
|
8) fp:fast vs fp:precise fails the test suite (same for gcc equivalents)
|
||||||
|
---------------------------------------------------------------------------------
|
||||||
|
As mentioned above, floating-point inaccuracies between differently optimized
|
||||||
|
builds are not considered regressions and all float comparisons done by the test
|
||||||
|
suite involve an epsilon. Changing floating-point optimizations can, however,
|
||||||
|
lead to *real* changes in the output data structure, such as different number
|
||||||
|
of vertices or faces, ... this violates one of our primary targets, that is
|
||||||
|
produce reliable and portable output. We're working hard on removing these
|
||||||
|
issues, but at the moment you have to live with few of them.
|
||||||
|
|
||||||
|
Currently, the regression database is build on Windows using MSVC8 with
|
||||||
|
fp:precise. This leads to a small number of failures with fp:fast and
|
||||||
|
virtally every build with gcc. Be patient, it will be fixed.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
Binary file not shown.
|
@ -0,0 +1,130 @@
|
||||||
|
#!/usr/bin/env python3
|
||||||
|
# -*- Coding: UTF-8 -*-
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Open Asset Import Library (ASSIMP)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
#
|
||||||
|
# Copyright (c) 2006-2010, ASSIMP Development Team
|
||||||
|
#
|
||||||
|
# All rights reserved.
|
||||||
|
#
|
||||||
|
# Redistribution and use of this software in source and binary forms,
|
||||||
|
# with or without modification, are permitted provided that the following
|
||||||
|
# conditions are met:
|
||||||
|
#
|
||||||
|
# * Redistributions of source code must retain the above
|
||||||
|
# copyright notice, this list of conditions and the
|
||||||
|
# following disclaimer.
|
||||||
|
#
|
||||||
|
# * Redistributions in binary form must reproduce the above
|
||||||
|
# copyright notice, this list of conditions and the
|
||||||
|
# following disclaimer in the documentation and/or other
|
||||||
|
# materials provided with the distribution.
|
||||||
|
#
|
||||||
|
# * Neither the name of the ASSIMP team, nor the names of its
|
||||||
|
# contributors may be used to endorse or promote products
|
||||||
|
# derived from this software without specific prior
|
||||||
|
# written permission of the ASSIMP Development Team.
|
||||||
|
#
|
||||||
|
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||||
|
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||||
|
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||||
|
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||||
|
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||||
|
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||||
|
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||||
|
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||||
|
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||||
|
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||||
|
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
"""Generate the regression database db.zip from the files in the <root>
|
||||||
|
/test/models directory. Older databases are overwritten with no prompt.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import sys
|
||||||
|
import os
|
||||||
|
import subprocess
|
||||||
|
import zipfile
|
||||||
|
|
||||||
|
import settings
|
||||||
|
import utils
|
||||||
|
|
||||||
|
# -------------------------------------------------------------------------------
|
||||||
|
def process_dir(d, outfile, file_filter):
|
||||||
|
""" Generate small dump records for all files in 'd' """
|
||||||
|
print("Processing directory " + d)
|
||||||
|
for f in os.listdir(d):
|
||||||
|
fullp = os.path.join(d, f)
|
||||||
|
if os.path.isdir(fullp) and not f == ".svn":
|
||||||
|
process_dir(fullp, outfile, file_filter)
|
||||||
|
continue
|
||||||
|
|
||||||
|
if file_filter(f):
|
||||||
|
for pp in settings.pp_configs_to_test:
|
||||||
|
print("DUMP " + fullp + "\n post-processing: " + pp)
|
||||||
|
outf = os.path.join(os.getcwd(), settings.database_name,
|
||||||
|
utils.hashing(fullp, pp))
|
||||||
|
|
||||||
|
cmd = utils.assimp_bin_path + " dump \"" + fullp + \
|
||||||
|
"\" " + outf + " -b -s " + pp
|
||||||
|
|
||||||
|
outfile.write("assimp dump "+"-"*80+"\n")
|
||||||
|
outfile.flush()
|
||||||
|
if subprocess.call(cmd, stdout=outfile, stderr=outfile, shell=False):
|
||||||
|
print("Failure processing " + fullp)
|
||||||
|
|
||||||
|
|
||||||
|
# -------------------------------------------------------------------------------
|
||||||
|
def make_zip():
|
||||||
|
"""Zip the contents of ./<settings.database_name>"""
|
||||||
|
zipout = zipfile.ZipFile(settings.database_name + ".zip", "w", zipfile.ZIP_DEFLATED)
|
||||||
|
for f in os.listdir(settings.database_name):
|
||||||
|
p = os.path.join(settings.database_name, f)
|
||||||
|
zipout.write(p, f)
|
||||||
|
if settings.remove_old:
|
||||||
|
os.remove(p)
|
||||||
|
|
||||||
|
if settings.remove_old:
|
||||||
|
os.rmdir(settings.database_name)
|
||||||
|
|
||||||
|
bad = zipout.testzip()
|
||||||
|
assert bad is None
|
||||||
|
|
||||||
|
|
||||||
|
# -------------------------------------------------------------------------------
|
||||||
|
def gen_db():
|
||||||
|
"""Generate the crash dump database in ./<settings.database_name>"""
|
||||||
|
try:
|
||||||
|
os.mkdir(settings.database_name)
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
outfile = open(os.path.join("..", "results", "gen_regression_db_output.txt"), "w")
|
||||||
|
(ext_list, err) = subprocess.Popen([utils.assimp_bin_path, "listext"],
|
||||||
|
stdout=subprocess.PIPE).communicate()
|
||||||
|
|
||||||
|
ext_list = str(ext_list).lower().split(";")
|
||||||
|
ext_list = list(filter(
|
||||||
|
lambda f: not f in settings.exclude_extensions, map(
|
||||||
|
lambda f: f.strip("b* \'"), ext_list)))
|
||||||
|
|
||||||
|
print(ext_list)
|
||||||
|
for tp in settings.model_directories:
|
||||||
|
process_dir(tp, outfile,
|
||||||
|
lambda x: os.path.splitext(x)[1] in ext_list)
|
||||||
|
|
||||||
|
|
||||||
|
# -------------------------------------------------------------------------------
|
||||||
|
if __name__ == "__main__":
|
||||||
|
utils.find_assimp_or_die()
|
||||||
|
gen_db()
|
||||||
|
make_zip()
|
||||||
|
|
||||||
|
print("="*60)
|
||||||
|
input("Press any key to continue")
|
||||||
|
|
||||||
|
# vim: ai ts=4 sts=4 et sw=4
|
||||||
|
|
|
@ -0,0 +1,253 @@
|
||||||
|
#!/usr/bin/env python3
|
||||||
|
# -*- Coding: UTF-8 -*-
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Open Asset Import Library (ASSIMP)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
#
|
||||||
|
# Copyright (c) 2006-2010, ASSIMP Development Team
|
||||||
|
#
|
||||||
|
# All rights reserved.
|
||||||
|
#
|
||||||
|
# Redistribution and use of this software in source and binary forms,
|
||||||
|
# with or without modification, are permitted provided that the following
|
||||||
|
# conditions are met:
|
||||||
|
#
|
||||||
|
# * Redistributions of source code must retain the above
|
||||||
|
# copyright notice, this list of conditions and the
|
||||||
|
# following disclaimer.
|
||||||
|
#
|
||||||
|
# * Redistributions in binary form must reproduce the above
|
||||||
|
# copyright notice, this list of conditions and the
|
||||||
|
# following disclaimer in the documentation and/or other
|
||||||
|
# materials provided with the distribution.
|
||||||
|
#
|
||||||
|
# * Neither the name of the ASSIMP team, nor the names of its
|
||||||
|
# contributors may be used to endorse or promote products
|
||||||
|
# derived from this software without specific prior
|
||||||
|
# written permission of the ASSIMP Development Team.
|
||||||
|
#
|
||||||
|
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||||
|
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||||
|
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||||
|
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||||
|
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||||
|
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||||
|
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||||
|
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||||
|
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||||
|
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||||
|
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
"""
|
||||||
|
Run the regression test suite using the settings from settings.py.
|
||||||
|
|
||||||
|
"""
|
||||||
|
|
||||||
|
import sys
|
||||||
|
import os
|
||||||
|
import subprocess
|
||||||
|
import zipfile
|
||||||
|
import collections
|
||||||
|
|
||||||
|
import settings
|
||||||
|
import utils
|
||||||
|
|
||||||
|
# -------------------------------------------------------------------------------
|
||||||
|
EXPECTED_FAILURE_NOT_MET, DATABASE_LENGTH_MISMATCH, \
|
||||||
|
DATABASE_VALUE_MISMATCH, IMPORT_FAILURE, \
|
||||||
|
FILE_NOT_READABLE, COMPARE_SUCCESS = range(6)
|
||||||
|
|
||||||
|
messages = collections.defaultdict(lambda: "<unknown", {
|
||||||
|
EXPECTED_FAILURE_NOT_MET:
|
||||||
|
"""Unexpected success during import\n\
|
||||||
|
\tReturn code was 0""",
|
||||||
|
|
||||||
|
DATABASE_LENGTH_MISMATCH:
|
||||||
|
"""Database mismatch: lengths don't match\n\
|
||||||
|
\tExpected: {0} Actual: {1}""",
|
||||||
|
|
||||||
|
DATABASE_VALUE_MISMATCH:
|
||||||
|
"""Database mismatch: """,
|
||||||
|
|
||||||
|
IMPORT_FAILURE:
|
||||||
|
"""Unexpected failure during import\n\
|
||||||
|
\tReturn code was {0}""",
|
||||||
|
|
||||||
|
FILE_NOT_READABLE:
|
||||||
|
"""Unexpected failure reading file""",
|
||||||
|
|
||||||
|
COMPARE_SUCCESS:
|
||||||
|
"""Results match archived reference dump in database\n\
|
||||||
|
\tNumber of bytes compared: {0}"""
|
||||||
|
})
|
||||||
|
|
||||||
|
outfilename_output = "run_regression_suite_output.txt"
|
||||||
|
outfilename_failur = "run_regression_suite_failures.csv"
|
||||||
|
# -------------------------------------------------------------------------------
|
||||||
|
class results:
|
||||||
|
|
||||||
|
""" Handle formatting of results"""
|
||||||
|
|
||||||
|
def __init__(self, zipin):
|
||||||
|
"""Init, given a ZIPed database """
|
||||||
|
self.failures = []
|
||||||
|
self.success = []
|
||||||
|
self.zipin = zipin
|
||||||
|
|
||||||
|
|
||||||
|
def fail(self, failfile, filename_expect, pp, msg, *args):
|
||||||
|
"""
|
||||||
|
Report failure of a sub-test
|
||||||
|
|
||||||
|
File f failed a test for pp config pp, failure notice is msg,
|
||||||
|
*args is format()ting args for msg
|
||||||
|
|
||||||
|
"""
|
||||||
|
print("[FAILURE] " + messages[msg].format(*args))
|
||||||
|
self.failures.append((failfile, filename_expect, pp))
|
||||||
|
|
||||||
|
|
||||||
|
def ok(self, f, pp, msg, *args):
|
||||||
|
"""
|
||||||
|
Report success of a sub-test
|
||||||
|
|
||||||
|
File f passed the test, msg is a happy success note,
|
||||||
|
*args is format()ing args for msg.
|
||||||
|
|
||||||
|
"""
|
||||||
|
print("[SUCCESS] " + messages[msg].format(*args))
|
||||||
|
self.success.append(f)
|
||||||
|
|
||||||
|
|
||||||
|
def report_results(self):
|
||||||
|
"""Write results to ../results/run_regression_suite_failures.txt"""
|
||||||
|
|
||||||
|
print("\n" + ('='*60) + "\n" + "SUCCESS: {0}\r\nFAILURE: {1}".format(
|
||||||
|
len(self.success), len(self.failures)) +
|
||||||
|
"\n" + ('='*60) + "\n")
|
||||||
|
|
||||||
|
with open(os.path.join('..', 'results',outfilename_failur), "wt") as f:
|
||||||
|
f.write("ORIGINAL FILE;EXPECTED DUMP\n")
|
||||||
|
f.writelines(map(
|
||||||
|
lambda x: x[0] + ' ' + x[2] + ";" + x[1] + "\n", self.failures))
|
||||||
|
|
||||||
|
if self.failures:
|
||||||
|
print("\nSee " + settings.results + "\\" + outfilename_failur
|
||||||
|
+ " for more details\n\n")
|
||||||
|
|
||||||
|
# -------------------------------------------------------------------------------
|
||||||
|
def mkoutputdir_andgetpath(fullpath, myhash, app):
|
||||||
|
outfile = os.path.join(settings.results, "tmp", os.path.split(fullpath)[1] + "_" + myhash)
|
||||||
|
try:
|
||||||
|
os.mkdir(outfile)
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
outfile = os.path.join(outfile, app)
|
||||||
|
return outfile
|
||||||
|
|
||||||
|
|
||||||
|
# -------------------------------------------------------------------------------
|
||||||
|
def process_dir(d, outfile_results, zipin, result):
|
||||||
|
shellparams = {'stdout':outfile_results, 'stderr':outfile_results, 'shell':False}
|
||||||
|
|
||||||
|
print("Processing directory " + d)
|
||||||
|
for f in os.listdir(d):
|
||||||
|
fullpath = os.path.join(d, f)
|
||||||
|
if os.path.isdir(fullpath) and not f == ".svn":
|
||||||
|
process_dir(fullpath, outfile_results, zipin, result)
|
||||||
|
continue
|
||||||
|
|
||||||
|
for pppreset in settings.pp_configs_to_test:
|
||||||
|
filehash = utils.hashing(fullpath, pppreset)
|
||||||
|
failure = False
|
||||||
|
try:
|
||||||
|
input_expected = zipin.open(filehash, "r").read()
|
||||||
|
# empty dump files indicate 'expected import failure'
|
||||||
|
if not len(input_expected):
|
||||||
|
failure = True
|
||||||
|
except KeyError:
|
||||||
|
#print("Didn't find "+fullpath+" (Hash is "+filehash+") in database")
|
||||||
|
continue
|
||||||
|
|
||||||
|
print("-"*60 + "\n " + os.path.realpath(fullpath) + " pp: " + pppreset)
|
||||||
|
|
||||||
|
outfile_actual = mkoutputdir_andgetpath(fullpath, filehash, "ACTUAL")
|
||||||
|
outfile_expect = mkoutputdir_andgetpath(fullpath, filehash, "EXPECT")
|
||||||
|
outfile_results.write("assimp dump "+"-"*80+"\n")
|
||||||
|
outfile_results.flush()
|
||||||
|
command = utils.assimp_bin_path + " dump \"" + fullpath + \
|
||||||
|
"\" " + outfile_actual + " -b -s " + pppreset
|
||||||
|
r = subprocess.call(command, **shellparams)
|
||||||
|
|
||||||
|
if r and not failure:
|
||||||
|
result.fail(fullpath, outfile_expect, pppreset, IMPORT_FAILURE, r)
|
||||||
|
continue
|
||||||
|
elif failure:
|
||||||
|
result.fail(fullpath, outfile_expect, pppreset, EXPECTED_FAILURE_NOT_MET)
|
||||||
|
continue
|
||||||
|
|
||||||
|
with open(outfile_expect, "wb") as s:
|
||||||
|
s.write(input_expected)
|
||||||
|
|
||||||
|
with open(outfile_actual, "rb") as s:
|
||||||
|
input_actual = s.read()
|
||||||
|
|
||||||
|
if len(input_expected) != len(input_actual):
|
||||||
|
result.fail(fullpath, outfile_expect, pppreset, DATABASE_LENGTH_MISMATCH,
|
||||||
|
len(input_expected), len(input_actual))
|
||||||
|
continue
|
||||||
|
|
||||||
|
outfile_results.write("assimp cmpdump "+"-"*80+"\n")
|
||||||
|
outfile_results.flush()
|
||||||
|
command = utils.assimp_bin_path + ' cmpdump "' + outfile_actual + '" "' \
|
||||||
|
+ outfile_expect + '"'
|
||||||
|
|
||||||
|
if subprocess.call(command, **shellparams) != 0:
|
||||||
|
result.fail(fullpath, outfile_expect, pppreset, DATABASE_VALUE_MISMATCH)
|
||||||
|
continue
|
||||||
|
result.ok(fullpath, pppreset, COMPARE_SUCCESS, len(input_expected))
|
||||||
|
|
||||||
|
# -------------------------------------------------------------------------------
|
||||||
|
def del_folder_with_contents(folder):
|
||||||
|
for root, dirs, files in os.walk(folder, topdown=False):
|
||||||
|
for name in files:
|
||||||
|
os.remove(os.path.join(root, name))
|
||||||
|
for name in dirs:
|
||||||
|
os.rmdir(os.path.join(root, name))
|
||||||
|
|
||||||
|
|
||||||
|
# -------------------------------------------------------------------------------
|
||||||
|
def run_test():
|
||||||
|
utils.find_assimp_or_die()
|
||||||
|
tmp_target_path = os.path.join(settings.results, "tmp")
|
||||||
|
try:
|
||||||
|
os.mkdir(tmp_target_path)
|
||||||
|
except OSError as oerr:
|
||||||
|
# clear contents if tmp folder exists already
|
||||||
|
del_folder_with_contents(tmp_target_path)
|
||||||
|
|
||||||
|
try:
|
||||||
|
zipin = zipfile.ZipFile(settings.database_name + ".zip",
|
||||||
|
"r", zipfile.ZIP_STORED)
|
||||||
|
|
||||||
|
except IOError:
|
||||||
|
print("Regression database ", settings.database_name,
|
||||||
|
".zip was not found")
|
||||||
|
return
|
||||||
|
|
||||||
|
res = results(zipin)
|
||||||
|
with open(os.path.join(settings.results, outfilename_output), "wt") as outfile:
|
||||||
|
for tp in settings.model_directories:
|
||||||
|
process_dir(tp, outfile, zipin, res)
|
||||||
|
|
||||||
|
res.report_results()
|
||||||
|
|
||||||
|
# -------------------------------------------------------------------------------
|
||||||
|
if __name__ == "__main__":
|
||||||
|
run_test()
|
||||||
|
input("Press any key to continue ...")
|
||||||
|
|
||||||
|
# vim: ai ts=4 sts=4 et sw=4
|
|
@ -0,0 +1,110 @@
|
||||||
|
#!/usr/bin/env python3
|
||||||
|
# -*- Coding: UTF-8 -*-
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Open Asset Import Library (ASSIMP)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
#
|
||||||
|
# Copyright (c) 2006-2010, ASSIMP Development Team
|
||||||
|
#
|
||||||
|
# All rights reserved.
|
||||||
|
#
|
||||||
|
# Redistribution and use of this software in source and binary forms,
|
||||||
|
# with or without modification, are permitted provided that the following
|
||||||
|
# conditions are met:
|
||||||
|
#
|
||||||
|
# * Redistributions of source code must retain the above
|
||||||
|
# copyright notice, this list of conditions and the
|
||||||
|
# following disclaimer.
|
||||||
|
#
|
||||||
|
# * Redistributions in binary form must reproduce the above
|
||||||
|
# copyright notice, this list of conditions and the
|
||||||
|
# following disclaimer in the documentation and/or other
|
||||||
|
# materials provided with the distribution.
|
||||||
|
#
|
||||||
|
# * Neither the name of the ASSIMP team, nor the names of its
|
||||||
|
# contributors may be used to endorse or promote products
|
||||||
|
# derived from this software without specific prior
|
||||||
|
# written permission of the ASSIMP Development Team.
|
||||||
|
#
|
||||||
|
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||||
|
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||||
|
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||||
|
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||||
|
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||||
|
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||||
|
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||||
|
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||||
|
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||||
|
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||||
|
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
"""Shared settings for the regression suite (bold builder and
|
||||||
|
test scripts rely on this)
|
||||||
|
|
||||||
|
"""
|
||||||
|
|
||||||
|
import os
|
||||||
|
# -------------------------------------------------------------------------------
|
||||||
|
# List of file extensions to be excluded from the regression suite
|
||||||
|
# File extensions are case insensitive
|
||||||
|
# -------------------------------------------------------------------------------
|
||||||
|
exclude_extensions = [".lws",".assbin",".assxml"]
|
||||||
|
|
||||||
|
# -------------------------------------------------------------------------------
|
||||||
|
# Post processing configurations to be included in the test. The
|
||||||
|
# strings are parameters for assimp_cmd, see assimp_cmd's doxydoc
|
||||||
|
# for more details.
|
||||||
|
|
||||||
|
# The defaults are (validate-data-structure is always enabled, for
|
||||||
|
# self-explanatory reasons :-):
|
||||||
|
#
|
||||||
|
# '-cfull' :apply all post processing except 'og' (optimize-scenegraph)
|
||||||
|
# '-og -om' :run optimize-scenegraph in combination with optimize-meshes.
|
||||||
|
# '-vds -jiv' :join-identical-vertices alone. This is a hotspot where
|
||||||
|
# floating-point inaccuracies can cause severe damage.
|
||||||
|
|
||||||
|
# As you can see, not all possible combinations of pp steps are covered -
|
||||||
|
# but at least each step is executed at least once on each model.
|
||||||
|
# -------------------------------------------------------------------------------
|
||||||
|
pp_configs_to_test = [
|
||||||
|
"-cfull",
|
||||||
|
"-og -om -vds",
|
||||||
|
"-vds -jiv"
|
||||||
|
]
|
||||||
|
# -------------------------------------------------------------------------------
|
||||||
|
# Name of the regression database file to be used
|
||||||
|
# gen_db.py writes to this directory, run.py checks against this directory.
|
||||||
|
# If a zip file with the same name exists, its contents are favoured to a
|
||||||
|
# normal directory, so in order to test against unzipped files the ZIP needs
|
||||||
|
# to be deleted.
|
||||||
|
# -------------------------------------------------------------------------------
|
||||||
|
database_name = "db"
|
||||||
|
|
||||||
|
# -------------------------------------------------------------------------------
|
||||||
|
# List of directories to be processed. Paths are processed recursively.
|
||||||
|
# -------------------------------------------------------------------------------
|
||||||
|
model_directories = [
|
||||||
|
os.path.join("..","models")
|
||||||
|
]
|
||||||
|
|
||||||
|
# -------------------------------------------------------------------------------
|
||||||
|
# Remove the original database files after the ZIP has been built?
|
||||||
|
# -------------------------------------------------------------------------------
|
||||||
|
remove_old = True
|
||||||
|
|
||||||
|
# -------------------------------------------------------------------------------
|
||||||
|
# Bytes to skip at the beginning of a dump. This skips the file header, which
|
||||||
|
# is currently the same 500 bytes header for both assbin, assxml and minidumps.
|
||||||
|
# -------------------------------------------------------------------------------
|
||||||
|
dump_header_skip = 500
|
||||||
|
|
||||||
|
# -------------------------------------------------------------------------------
|
||||||
|
# Directory to write all results and logs to. The dumps pertaining to failed
|
||||||
|
# tests are written to a subfolder of this directory ('tmp').
|
||||||
|
# -------------------------------------------------------------------------------
|
||||||
|
results = os.path.join("..","results")
|
||||||
|
|
||||||
|
|
||||||
|
# vim: ai ts=4 sts=4 et sw=4
|
|
@ -0,0 +1,108 @@
|
||||||
|
#!/usr/bin/env python3
|
||||||
|
# -*- Coding: UTF-8 -*-
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Open Asset Import Library (ASSIMP)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
#
|
||||||
|
# Copyright (c) 2006-2010, ASSIMP Development Team
|
||||||
|
#
|
||||||
|
# All rights reserved.
|
||||||
|
#
|
||||||
|
# Redistribution and use of this software in source and binary forms,
|
||||||
|
# with or without modification, are permitted provided that the following
|
||||||
|
# conditions are met:
|
||||||
|
#
|
||||||
|
# * Redistributions of source code must retain the above
|
||||||
|
# copyright notice, this list of conditions and the
|
||||||
|
# following disclaimer.
|
||||||
|
#
|
||||||
|
# * Redistributions in binary form must reproduce the above
|
||||||
|
# copyright notice, this list of conditions and the
|
||||||
|
# following disclaimer in the documentation and/or other
|
||||||
|
# materials provided with the distribution.
|
||||||
|
#
|
||||||
|
# * Neither the name of the ASSIMP team, nor the names of its
|
||||||
|
# contributors may be used to endorse or promote products
|
||||||
|
# derived from this software without specific prior
|
||||||
|
# written permission of the ASSIMP Development Team.
|
||||||
|
#
|
||||||
|
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||||
|
# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||||
|
# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
|
||||||
|
# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
|
||||||
|
# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
|
||||||
|
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
|
||||||
|
# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
|
||||||
|
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
|
||||||
|
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||||
|
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||||
|
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
"""Shared stuff for the gen_db and run scripts """
|
||||||
|
|
||||||
|
# -------------------------------------------------------------------------------
|
||||||
|
def hashing(file,pp):
|
||||||
|
""" Map an input file and a postprocessing config to an unique hash.
|
||||||
|
|
||||||
|
The hash is used to store the item in the database. The buildin
|
||||||
|
hash() function is used. The hash is a string.
|
||||||
|
|
||||||
|
"""
|
||||||
|
return hex(hash(file+":"+pp))
|
||||||
|
|
||||||
|
|
||||||
|
assimp_bin_path = None
|
||||||
|
# -------------------------------------------------------------------------------
|
||||||
|
def find_assimp_or_die():
|
||||||
|
"""Find assimp_cmd's binary for the current platform.
|
||||||
|
|
||||||
|
The path to the binary is stored in assimp_bin_path, the process
|
||||||
|
is aborted if it can't be found.
|
||||||
|
|
||||||
|
"""
|
||||||
|
|
||||||
|
import os
|
||||||
|
import platform
|
||||||
|
import sys
|
||||||
|
|
||||||
|
def locate_file(f_list):
|
||||||
|
for f in f_list:
|
||||||
|
try:
|
||||||
|
fl = open(f,"rb")
|
||||||
|
except IOError:
|
||||||
|
continue
|
||||||
|
fl.close()
|
||||||
|
return f
|
||||||
|
return None
|
||||||
|
|
||||||
|
global assimp_bin_path
|
||||||
|
if os.name == "nt":
|
||||||
|
if platform.machine() == "x86":
|
||||||
|
search = [os.path.join("..","..","bin","assimpcmd_release-dll_Win32","assimp.exe"),
|
||||||
|
os.path.join("..","..","bin","x86","assimp")]
|
||||||
|
|
||||||
|
else: # amd64, hopefully
|
||||||
|
search = [os.path.join("..","..","bin","assimpcmd_release-dll_x64","assimp.exe"),
|
||||||
|
os.path.join("..","..","bin","x64","assimp")]
|
||||||
|
|
||||||
|
elif os.name == "posix":
|
||||||
|
search = [os.path.join("..","..","bin","gcc","assimp"),
|
||||||
|
os.path.join("/usr","local","bin")]
|
||||||
|
else:
|
||||||
|
print("Unsupported operating system")
|
||||||
|
sys.exit(-5)
|
||||||
|
|
||||||
|
assimp_bin_path = locate_file(search)
|
||||||
|
if assimp_bin_path is None:
|
||||||
|
print("Can't locate assimp_cmd binary")
|
||||||
|
sys.exit(-5)
|
||||||
|
|
||||||
|
print("Located assimp/assimp_cmd binary at ",assimp_bin_path)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
find_assimp_or_die()
|
||||||
|
|
||||||
|
# vim: ai ts=4 sts=4 et sw=4
|
Loading…
Reference in New Issue