rtedge — ray-traces a model, detects edges, and writes a BRL-CAD color image (.pix) file


rtedge [options...] model.g objects... < model.pix


rtedge operates on the indicated objects in the input model.g and produces a BRL-CAD .pix file that indicates the 'edges' in the model file. rtedge produces images by drawing a boundary whenever a change in region_id is detected. It also recognizes and portrays abrupt changes in surface curvature and changes in surface height. This permits the recognition of pits, protrusions, and changes in surface curvature.

The orientation of the rays to be fired may be specified by the -a and -e options, in which case the model will be autosized, and the grid will be centered on the centroid of the model, with ray spacing chosen to span the entire set of objects. Alternatively, with the -M option, a transformation matrix may be provided on standard input which maps model-space to view-space. In this case, the grid ranges from -1.0 < = X,Y < = +1.0 in view space, with the size of the grid (number of rays fired) specified with the -s option. This option is most useful when rtedge is being invoked from a shell script created by an mged(1) saveview command.

The following options are recognized:

-. #

selects the factor used in NUgrid algorithm. Default is 1.5 .

-, #

selects which space partitioning algorithm to use

-a #

selects auto-sizing, and gives the view azimuth in degrees. Used with -e and conflicts with -M .

-b "# #"

is sometimes useful for debugging. A single ray is fired at the pixel coordinates specified. This is best when accompanied by debugging options. Note that the standard getopt(3) routine only allows options to have one parameter, so the X and Y pixel coordinates need to be combined into one string parameter by enclosing them in double quotes. This option must follow any viewing parameters.

-d #

sets flag for reporting of pixel distances: 1 for on, 0 (default) for off. (Each pixel is represented by 3 bytes of color information followed by 8 bytes for a double, in IEEE format, which gives the distance to the ray's first hit. If the ray doesn't hit, a zero distance is reported by default which causes rt(1) to write only the color information. See pix(5) and pixd(5).)

-e #

selects auto-sizing, and gives the view elevation in degrees. Used with -a and conflicts with -M .

-f #

sets the expected playback rate in frames-per-second; the default is 30. Several time-evolving shaders such as fire and grass need to know the time at which each frame and pixel occur to drive their evolution.

-g #

sets the grid cell width, in millimeters. Ordinarily, the cell width is computed by dividing the view size by the number of pixels of width, so this option need not be used. To obtain exactly 1 inch ray spacing, specify -g25.4 . If the grid cell width is specified and the grid cell height (which would be set using -G) is not specified, the grid cell height defaults to be the same as the width.

-m density,r,g,b

enables an exponential shading of the void between the eye and the first hit location. This allows a cheap simulation of atmospheric haze with lower performance impact than explicit modeling of the air volume. The haze is computed using the form:

f = e^(-distance * density)

g = 1.0 - f

color = color * f + hazeRGB * g

The color values are optional. If not specified, they will have the passable values 0.8 .9 .99 .

Example usage: -m '2.5e-8,.75,.95,.99'


enables incremental mode processing. (First, a single ray is fired. Then, the screen is subdivided into four parts by evenly subdividing in the vertical and horizontal directions. The previously fired ray was the lower left corner; three more rays are fired. This process recurses until full resolution has been reached. Assumes square image. Also assumes fast path between CPU and display hardware.)

-j xmin,ymin,xmax,ymax

enables processing of just a sub-rectangle, located within the rectangle (0,0) (width-1,height-1), and whose screen coordinates (lower left and upper right corners) are provided in the argument. Ray-tracing will be limited to just those pixels within the sub-rectangle, and only those pixels will be transmitted to the framebuffer. The corner pixels (xmin,ymin) and (xmax,ymax) are included in the sub-rectangle. This option exists primarily for mged(1) to allow sweeping out a rectangular portion of the view for ray-tracing.

-k xdir,ydir,zdir,dist

enables the use of a cutting plane. The xdir, ydir, zdir, and dist define a plane (equivalent to a halfspace primitive with the same parameters). The vector (xdir, ydir, zdir) is normal to the plane and points away from the space to be subtracted. The dist parameter specifies the distance from the origin to the plane in the normal direction. The resulting image approximates subtracting the halfspace from the objects being raytraced. Note that this is not exactly the same as actually subtracting the halfspace, but it is faster. If explicit lights are used, objects that are cut away by the cutting plane will still cast shadows.

-l #

selects lighting model. Default is 0. Model 0 is a full lighting model with the ability to implement Phong shading, transparent and reflective objects, shadow penumbras, texture maps, etc. In addition to ambient light, a small amount of light is supplied from eye position. All objects in the active model space with a material property string of "light" represent additional light sources (up to 16 are presently permitted), and shadow computations will be initiated automatically. (This mechanism is subject to further change.) Model 1 is a diffuse-lighting model only, and is intended for debugging. Model 2 displays the surface normals as colors, and is useful for examining curvature and surface orientation. Model 3 is a three-light diffuse-lighting model, and is intended for debugging. Model 4 is a curvature debugging display, showing the inverse radius of curvature. Model 5 is a curvature debugging display, showing the principal direction vector. Model 6 is for debugging the UV parameterization used for surface texturing. It is exactly the same as applying the texture 'testmap' to every region. The U parameter is displayed in the red channel and the V parameter is displayed in the blue channel. Model 7 is for using photon mapping to simulate global illumination and caustics in a given scene. Options to photon mapping are passed through the -l parameter in the following format:

-l7,Num_Photons,Caustics_Percent,Irradiance_Rays,Angular_Tolerance, Random_Seed,Importance_Mapping_Toggle,Irradiance_Hypersampling_Toggle, Visual_Irradiance_Toggle,Scale_Lumens,Load_Save_File

Example: -l7,16384,0,10,60.0,0,0,0,0,1.0,

Note: If just -l7 is used, then the default values above will be applied. Light sources must also be explicitly defined with lumens defined for each source, i.e., default light sources will not work.

-n #

defines the height of the image as a number of scanlines.

-o output.pix

specifies the name of a file to store the pix(5) format output. The pix-fb(1) utility can be used later to display ".pix" files. Note that using -o disables parallel processing.

-p #

sets the perspective, in degrees, with 0 < = # < 180


requests that overlapping regions be reported (on by default); opposite of -R .

-s #

specifies the number of rays to fire in the X and Y directions on a square grid. The default size is 512 (i.e., 512 x 512).


causes reversal of the order of grid traversal (default is not to do that)

-u units

specifies units, with "model" being used for the units of the local model

-v #

sets the verbosity bit vector flags for rtedge . Use -v 0 for no output or -v fff for all available output. See include/brlcad/rtprivate.h for various bit definitions.

-w #

defines the width of each scanline in pixels.

-x #

sets the librt(3) debug flags to the given hexadecimal bit vector. See raytrace.h for the bit definitions. (Notice that -! # sets the libbu(3) debug flags to the given hexadecimal bit vector. See bu.h for the bit definitions.)

-A #

Select angle for shading. Default is 5.0 degrees; 89.0 will produce an image where only steep drops and rises are shaded.


turns on the "benchmark" flag. This causes all intentional random effects such as ray dither, color dither, etc., to be turned off to allow benchmark timing comparison and benchmark result comparison.

-C #/#/#

sets the background color to the RGB value #/#/#, where each # is in the range of 0 to 255. All non-background colors will be dithered away from this value.

-D #

This is the "desired frame" flag, specifying the frame number to start with. Used with animation scripts; specify the desired ending (kill-after) frame with -K # .

-E #

sets the distance from the eye point to the center of the model RPP; default is the square root of 2. Only useful with auto-sizing, and conflicts with -M .

-F framebuffer

indicates that the output should be sent to the indicated framebuffer. See libfb(3) for more details on the specification of a framebuffer.

-G #

sets the grid cell height, in millimeters. If the grid cell height is specified and the grid cell width (which would be set using -g) is not specified, the grid cell width defaults to be the same as the height.

-H #

This is the "hypersample" flag, specifying number of extra rays to fire for each pixel to obtain more accurate results without needing to store the extra pixels. Automatically invokes -J 1. The final colors of all rays are averaged together. Better results can be obtained by simply increasing the resolution, and decimating the results with a filter such as pixhalve(1).


turns on interactive mode. If this flag is off and your image is "large" (> = 256 x 256), it will lower the priority of the rt process.

-J #

This is the "jitter" flag, which is a bit vector. If one (-J 1), it randomizes the point from which each ray is fired by +/- one half of the pixel size. Useful if doing your own post-filtering for antialiasing, or to eliminate systematic errors. This has been useful in obtaining more accurate results in a volume-calculating raytrace as well. If two (-J 2), it shifts the origin of the output frame once for each frame. The shift is along a circular path in the plane of the frame, with a period of ten seconds, and a radius of +/- 1/4 pixel width. If three (-J 3), it performs both operations.

-K #

sets the final (kill-after) frame number. Used with animation scripts in conjunction with -D # .


causes reading of animation (model2view) matrix or animation script from standard input. Conflicts with -a and -e options.

-N #

sets the NMG_debug flags to the given hexadecimalbit vector. See h/nmg.h for the bit definitions.

-O output.dpix

specifies the name of a file to store the double-precision floating-point version of the image. dpix-pix(1) can be used to later convert the file to pix(5) format output. (Deferred implementation)

-P #

specifies the maximum number of processors (in a multi-processor system) to be used for this execution. The default is system dependent. On "dedicated" multi-processors, such as workstations and super-minis, the default is usually set for the maximum number of processors, while on shared multi-processors, such as SuperComputers, usually just one processor is used by default.

-Q x,y

selects pixel ray for query with debugging. The option is used to specify a single pixel for which the debug flags should be turned on. The computation of all other pixels will be performed without debugging. This is similar to the -b option, except that it does not limit rendering to the single ray. The entire image is computed. This option is useful in debugging problems which arise from specific computation sequences (such as the order of random numbers previously drawn).


requests that overlapping regions not be reported; opposite of -r .


turns on stereo viewing. The left-eye image is drawn in red, and the right-eye image is drawn in blue.

-T # or -T #,# or -T #/#

sets the calculation tolerances used by librt(3). If only one number is given, it specifies the distance tolerance. Any two entities closer than this distance are considered to be the same. The default distance tolerance is 0.005mm. If the second number is given, it specifies the value of the dot product below which two lines are to be considered perpendicular. This number should be near zero, but in any case should be less than 1. The default perpendicular tolerance is 1e-6. If both calculation tolerances are provided, they shall be separated by either a slash ("/") or a comma (","), with no white space between them. For example, the default tolerances would be specified as -T0.005/1e-6 on the command line.

-U #

sets the Boolean variable use_air to the given value. The default is 0, which ignores regions that have non-zero aircode values.

-V #

sets the view aspect. This is the ratio of width over height and can be specified as either a fraction or a colon-separated ratio. For example, the NTSC aspect ratio can be specified by either -V1.33 or -V4:3 .


inverts the foreground and background colors, changing the default from white lines on a black background to black lines on a white background

-X #

turns on the rt(1) program debugging flags. See raytrace.h for the meaning of these bits.

-! #

turns on the libbu(3) library debugging flags. See bu.h for the meaning of these bits. In particular, specifying -!1 will cause bu_bomb() to dump core, rather than logging the error and exiting. This is useful with dbx(1) or gdb(1) for debugging. (Notice that -x # sets the libbt(3) debug flags to the given hexadecimal bit vector. See raytrace.h for the bit definitions.)

-+ #

specifies (setting argument to "t") that output is NOT binary (default is that it is). -+ is otherwise not implemented.


Set special rtedge configuration variables.

The rtedge program is a simple front-end to librt(3).


rtedge replaces lgt(1). In addition to the simple white edges on a black background (or black on white) provided by lgt, rtedge allows arbitrary color combinations. It also allows rendering into an existing framebuffer and occlusion detection between two pieces of geometry. To configure its behavior, rtedge makes copious use of the -c option.

Note that this approach is necessitated by the fact that librt(3) has used up nearly the entire alphabet. A transition to GNU-style long option names is planned.

-c "set foreground=#,#,#"


-c "set fg=#,#,#"

set the color of the foreground (edges) to the given r,g,b triple. The default is white (255,255,255).

-c "set background=#,#,#"


-c "set bg=#,#,#"

set the color of the background to the r,g,b triple. The default is near-black (0,0,1).

-c "set region_colors=#"


-c "set rc=#"

determine whether to use the material color assigned to an object as the edge color. Valid values are 1 (on) and 0 (off). The default is off. Using region colors overrides any foreground color setting.

-c "set detect_regions=#"


-c "set dr=#"

determine whether the boundary between two BRL-CAD regions is considered to be an edge. Valid values are 1 (on) and 0 (off). The default is off.

-c "set detect_distance=#"


-c "set dd=#"

determine whether a difference in hit distance between adjacent pixels defines an edge. Valid values are 1 (on) and 0 (off). The default is on.

-c "set detect_normals"


-c "set dn=#"

determine whether a difference in surface normals between adjacent pixels defines an edge. Valid values are 1 (on) and 0 (off). The default is on.

-c "set detect_ids"


-c "set di=#"

determine whether a difference in component id numbers between adjacent pixels defines an edge. Valid values are 1 (on) and 0 (off). The default is on.

-c "set overlaymode=#"


-c "set ov=#"

configure the overlay mode to be either on (1) or off (0). Overlay mode allows rtedge to write into an existing framebuffer. The framebuffer is specified with the -F option.

-c "set max_dist=#"


-c "set md=#"

override the maximum hit distance of neighboring pixels before being declared an edge. The default value is computed based on the size of the output image, approximately 1000 for the default image/fb size.

-c "set antialias"


-c "set aa"

configure whether to perform antialiasing on edges. Valid values are 1 (on) and 0 (off). The default is off.

-c "set both_sides"


-c "set bs"

configure whether to render both sides of an edge. result are edges twice as thick but that work well with antialiasing. Valid values are 1 (on) and 0 (off). The default is off.


rtedge has the ability to perform occlusion checking between two sets of BRL-CAD objects from the same database. Occlusion checking determines which set of geometry is closer to the eyepoint and thus should be rendered. The first set is that specified in the basic object list (after the database). This set will be rendered using edge detection. The second is specified using a -c option.

Occlusion checking is most handy when combining rtedge renderings with rt renderings. The syntax for such compositions is daunting, but the results are cool.

-c"set occlusion_objects=\"obj1 obj2 ... objN\


-c"set oo=\"obj1 obj2 ... objN\

specify the second set of geometry.

-c"set occlusion_mode=#


-c"set om=#

determine how rtedge behaves when performing occlusion checking. There are three valid modes. Mode 2 is the default.

mode 1 - An edge detected in the first set of geometry is written to the framebuffer if and only if it occludes the second set of geometry. The edge is colored according to the foreground or region colors options.

mode 2 - All pixels that hit the first set of geometry that are not occluded by the second set are written to the framebuffer. Edges are rendered in the foreground or region color; non-edges are rendered in the background color.

mode 3 - Like mode 2 except non-edge pixels are dithered to make the geometry semi-transparent.


rtedge can be somewhat complicated to configure. These examples illustrate both simple and complex examples. Hopefully they will be enough to get you started. These examples use the havoc.g target description that may be found in the source distribution.

Note that in practice it will usually be easiest to use the saveview command in mged to produce a script and then modify that script to run rtedge .

Example 1. rtedge rendering of havoc.g

rtedge -s 1024 -o new.pix havoc.g havoc

results in a 1024 by 1024 BRL-CAD image file showing the edges on a Havoc helicopter. The default settings are used, so the image has white edges on a black background. Edges exist where there are differences in component id, hit distance, surface normal, or between hits and misses.

The command rtedge -s1024 -o new.pix -c"set fg=0,255,0 bg=255,0,0" havoc.g havoc changes the edge color to be green and the background color to be red.

The command rtedge -s1024 -o new.pix -c"set rc=1 dr=1" havoc.g havoc activates region detection and uses the region colors for the edges.

The remaining examples will illustrate how to use rtedge in concert with rt to produce interesting images. When merging rtedge and rt images, it is best to use saveview scripts. In mged, configure your view, save it, and then open the file in an editor. Duplicate the rt command block. After duplicating the block, change the second to rtedge .

This script will overlay bright orange edges on the Havoc.

rt -M -s1280 -o new.pix -C255/255/255\
 2>> example1.log\
viewsize 8.000e+03;
orientation 2.4809e-01 4.7650e-01 7.4809e-01 3.8943e-01;
eye_pt 2.2146e+04 7.1103e+03 7.1913e+03;
start 0; clean;


rtedge -M -s1280 -o new.pix \
 -c"set dr=1 dn=1 ov=1"\
 -c"set fg=255,200,0" \
 2>> example1.log\

 << EOF
viewsize 8.000e+03;
orientation 2.4809e-01 4.7650e-01 7.4809e-01 3.8943e-01;
eye_pt 2.2146e+04 7.1103e+03 7.1913e+03;
start 0; clean;


Finally, this script will render the Havoc weapon systems in full color, render the edges on the remainder of the aircraft in black, and render the non-edges in dithered gray to make them semi-transparent. Note that when specifying the occlusion objects, the delimiting quotes must be escaped.


rt -M -s1280 -o new.pix -C255/255/255 \
 2>> example2.log\
 << EOF
viewsize 8.000e+03;
orientation 2.4809e-01 4.7650e-01 7.4809e-01 3.8943e-01;
eye_pt 2.2146e+04 7.1103e+03 7.1913e+03;
start 0; clean;


rtedge -M -s1280 -o new.pix \
 -c"set dr=1 dn=1 om=3"\
 -c"set fg=0,0,0 bg=200,200,200"\
 -c"set oo=\"weapons\" "\
 havoc_front havoc_middle havoc_tail landing_gear main_rotor\
 2>> example2.log\

 << EOF
viewsize 8.000e+03;
orientation 2.4809e-01 4.7650e-01 7.4809e-01 3.8943e-01;
eye_pt 2.2146e+04 7.1103e+03 7.1913e+03;
start 0; clean;


In general, if you are mixing rt and rtedge renderings, and the objects being rendered are different, occlusion checking should be used.


mged(1), rt(1), fbserv(1), librt(3)


Numerous error conditions are possible, usually due to errors in the geometry database. Descriptive messages are printed on stderr (file descriptor 2).


Ronald Anthony Bowers

Michael John Muuss


This software is Copyright (c) 2001-2016 United States Government as represented by the U.S. Army Research Laboratory.


Most deficiencies observed while using the rtedge program are a consequence of problems in librt(3).


Reports of bugs or problems should be submitted via electronic mail to <>.