From: Geoff Sokoll 
To: iccon-misc@loki.sdrc.com 
Date: Fri, 21 Feb 1997 01:20:29 GMT 

Some comments on the benchmarking experience:

I have recently completed a series of benchmarking exercises, comparing
our existing hardware to some new systems.  The first step in this 
process was recording exactly what we spent most of our time doing. 
From this, I compiled the following list of typical tasks that were on 
the critical path (ie either time consuming or repeated often, or both):

  previewing and generating free meshes
  creating and displaying element groups
  linear and non-linear static and normal mode FEA solves
  importing and exporting universal files
  scaling, combining and sorting FEA result sets
  displaying FEA results
  screen redisplays
 
Obviously, we spend most of our time in the Simulation task, doing FEA. I
wrote a program file to carry out most of the above tasks on several models
of varying size, recording the average times taken.  This enabled us to make
an estimate of the daily time savings to be expected with one of the new
machines.

With the benefit of hindsight, I would make the following recommendations to
anyone considering evaluating a machine:

1)  Look around first for existing benchmarks (SPEC95, STREAM etc) to enable
you to narrow down your choices
2)  Evaluate what your bottlenecks are. Those seemingly trivial tasks, if
repeated 100 times a day, may not be so trivial.
3)  Get a loan of the machine you wish to test, from the hardware vendor,
for AT LEAST a week, preferably more - you'll need it.
4)  Run your benchmark tests using models and tasks representative of what
you spend most of your time working with.  Trying to estimate solution times
for a non-linear 100,000 dof FEM model from the 30 seconds or so taken to
carry out the bullet-hole block benchmark is futile.

Regarding setting up a repository of benchmark results, I think that given
the wide range of possible tasks to perform within I-DEAS and the large
number of possible system configurations, this might be too big for someone
to handle on a volunteer basis.

If anyone is interested, I would be willing to share some of the program
files I wrote to conduct our own benchmarking exercises (heavily biased
towards FE analysis).

Geoff Sokoll
WBM Pty Ltd, Australia
wbmmech@powerup.com.au

In order to make the program files work for everyone, they must be platform independent. The best way ensure that the program you create will run on other platforms is to follow these simple guidelines :

	Start with a new model file.
	Build the geometry from scratch or
	Read the geometry from a universal file.
	Avoid mouse picks from the graphics window.
	Key-in labels whenever possible.
	Turn Menu Mnemonics On.
	Use icons or select from the menus.
	Run and Rerun the program file to verify that it works!

In order to accurately record the time to execute the program file, we'll use I-DEAS programmability. The following program file code is from Rollo Cain.

At the beginning of the program file... turn echo off, initialize a "Stopwatch", and (usually) turn off view transitions:

c : =============
K : #echo none
K :  #total=stop_watch(0)
K :  #c_time=cpu_time(0)
c :
K : /O P
K : P 5;
K : VT OF
K : OKAY
K : OKAY
c : =============

At the end of the program file, record the "Stopwatch" time and restore view transitions and echo:

c : =============
K :  #output  " Benchmark Test has Finished "
K :  #total=stop_watch(total)
K :  #c_time=cpu_time(c_time)
K :  #output  " "
K :  #output  "*****     Benchmark Test Results     *****"
K :  #output  " Total Elapsed time =  ",total," seconds."
K :  #output  " Total CPU time =  ",c_time," seconds."
K :  #open result result.log
K :  #write result  "Benchmark Test Results :"
K :  #write result  "  Total Elapsed time  =  ",total," seconds."
K :  #write result  "  Total CPU time  =  ",c_time," seconds."
K :  #close result
K :  #output  "Results written to ""./results.log""  "
c :
K : /O P
K : P 5;
K : VT ON
K : OKAY
K : OKAY
K : #echo all
c : =============

Updated: 01 January 2005

® Copyright © 1996-