Home | History | Annotate | Download | only in tools
      1 #
      2 # CDDL HEADER START
      3 #
      4 # The contents of this file are subject to the terms of the
      5 # Common Development and Distribution License (the "License").
      6 # You may not use this file except in compliance with the License.
      7 #
      8 # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
      9 # or http://www.opensolaris.org/os/licensing.
     10 # See the License for the specific language governing permissions
     11 # and limitations under the License.
     12 #
     13 # When distributing Covered Code, include this CDDL HEADER in each
     14 # file and include the License file at usr/src/OPENSOLARIS.LICENSE.
     15 # If applicable, add the following below this CDDL HEADER, with the
     16 # fields enclosed by brackets "[]" replaced with your own identifying
     17 # information: Portions Copyright [yyyy] [name of copyright owner]
     18 #
     19 # CDDL HEADER END
     20 #
     21 #
     22 # Copyright 2009 Sun Microsystems, Inc.  All rights reserved.
     23 # Use is subject to license terms.
     24 
     25 
     26 This directory contains the tools used to do a full build of the
     27 OS/Net workspace.  They usually live in the /opt/onbld directory on build
     28 machines. From here, 'make install' will build and install the tools
     29 in $ROOT/opt/onbld. If you like, 'make pkg' will build the SUNWonbld
     30 package in $(PKGARCHIVE). Installing that package will populate the
     31 /opt/onbld directory, and create a root account for building called 'gk',
     32 which uses csh and has a home directory of /opt/onbld/gk. You can
     33 use this account to do full builds with 'nightly'. You don't have to,
     34 but the 'gk' account has the path setup properly, has a .make.machines
     35 file for dmake, and has a .login that sets up for dmake.
     36 
     37 Layout of /opt/onbld
     38 --------------------
     39 
     40 /opt/onbld/etc/abi
     41 	contains Solaris ABI database (ABI_*.db) and exceptions
     42 	for ABI Auditing tool (interface_check, interface_cmp).
     43 
     44 /opt/onbld/gk
     45 	gk account's home directory.
     46 
     47 /opt/onbld/bin
     48 	basic bin directory - contains scripts.
     49 
     50 /opt/onbld/bin/${MACH}
     51 	architecture-specific bin directory for binaries.
     52 
     53 /opt/onbld/env
     54 	build environment files.
     55 
     56 /opt/onbld/lib
     57 	libraries used by the build tools.
     58 
     59 /opt/onbld/lib/python
     60 	python modules used by the build tools.
     61 
     62 /opt/onbld/lib/python/onbld/hgext
     63 	Mercurial extensions.
     64 
     65 /opt/onbld/man
     66 	rudimentary man pages for some of the tools.
     67 
     68 
     69 Tool Summary
     70 ------------
     71 
     72 bfu
     73 	bonwick/faulkner upgrade. Loads a set of cpio archives created
     74 	by 'mkbfu' onto a machine, either live or on alternate root
     75 	and /usr filesystems. Attempts to preserve important files,
     76 	but may require manual intervention before reboot to resolve
     77 	changes to preserved files.
     78 
     79 bfuld
     80 	Used by bfu to survive getting a new runtime linker when extracting
     81 	new cpio archives onto a live system. Patches binaries to use
     82 	a saved runtime linker in /tmp during the bfu process.
     83 	Not run by anything but bfu.
     84 
     85 bldenv
     86 	companion to 'nightly.' Takes the same environment file you
     87 	used with 'nightly,' and starts a shell with the environment
     88 	set up the same way as 'nightly' set it up. This is useful
     89 	if you're trying to quickly rebuild portions of a workspace
     90 	built by 'nightly'. 'ws' should not be used for this since it
     91 	sets the environment up differently and may cause everything
     92 	to rebuild (because of different -I or -L paths).
     93 
     94 build_cscope
     95 	builds cscope databases in the uts, the platform subdirectories
     96 	of uts, and in usr/src. Uses cscope-fast.
     97 
     98 cdm 
     99 	A Mercurial extension providing various commands useful for ON
    100 	development
    101 
    102 check_rtime
    103 	checks ELF attributes used by ELF dynamic objects in the proto area.
    104 	Used by 'nightly's -r option, to check a number of ELF runtime
    105 	attributes for consistency with common build rules.  nightly uses
    106 	the -o option to simplify the output for diffing with previous
    107 	build results.  It also uses the -i option to obtain NEEDED and RUNPATH
    108 	entries, which help detect changes in software dependencies and makes
    109 	sure objects don't have any strange runpaths like /opt/SUNWspro/lib.
    110 
    111 checkproto
    112 	Runs protocmp and protolist on a workspace (or uses the environment
    113 	variable CODEMGR_WS to determine the workspace). Checks the proto area
    114 	against the packages.
    115 
    116 codereview
    117 	Given two filenames, creates a postscript file with the file 
    118 	differences highlighted.
    119 
    120 codesign
    121 	Tools for signing cryptographic modules using the official
    122 	Sun release keys stored on a remote signing server. This
    123 	directory contains signit, a client program for signing
    124 	files with the signing server; signproto, a shell script
    125 	that finds crypto modules in $ROOT and signs them using
    126 	signit; and codesign_server.pl, the code that runs on the
    127 	server. The codesign_server code is not used on an ON
    128 	build machine but is kept here for source control purposes.
    129 
    130 copyrightchk
    131 	Checks that files have appropriate SMI copyright notices.
    132 	Primarily used by wx
    133 
    134 cscope-fast
    135 	The fast version of cscope that we use internally. Seems to work,
    136 	but may need more testing before it's placed in the gate. The source
    137 	just really needs to be here.
    138 	
    139 cstyle
    140 	checks C source for compliance with OS/Net guidelines.
    141 
    142 ctfconvert
    143 	Convert symbolic debugging information in an object file to the Compact
    144 	ANSI-C Type Format (CTF).
    145 
    146 ctfdump
    147 	Decode and display CTF data stored in a raw file or in an ELF file.
    148 
    149 ctfmerge
    150 	Merge the CTF data from one or more object files.
    151 
    152 depcheck
    153 	A tool to try an assess the dependencies of executables.  This tool 
    154 	is not a definitive dependency check, but it does use "strings" and 
    155 	"ldd" to gather as much information as it can.  The dependency check
    156 	tool can handle filenames and pkgnames.  Before using the dependency
    157 	checker you must build a database which reflects the properties and
    158 	files in your system.
    159 
    160 elfcmp
    161 	Compares two ELF modules (e.g. .o files, executables) section by
    162 	section.  Useful for determining whether "trivial" changes -
    163 	cstyle, lint, etc - actually changed the code.  The -S option
    164 	is used to test whether two binaries are the same except for
    165 	the elfsign signature.
    166 
    167 elfsign
    168 	Built from the same sources as the shipped elfsign(1), this
    169 	version is used in nightly -t builds to assure that the signing
    170 	process and format is the same as will be used on the target
    171 	system.
    172 
    173 elfsigncmp
    174 	This script can be used in lieu of elfsign during a build.
    175 	It uses elfsign to sign a copy of the object and elfcmp -S to
    176 	verify that the signing caused no damage before updating
    177 	the object to be signed.
    178 	
    179 find_elf
    180 	Search a directory tree for ELF objects, and produce one line of
    181 	output per object. Used by check_rtime and interface_check to locate
    182 	the objects to examine.
    183 
    184 findunref
    185 	Finds all files in a source tree that have access times older than a
    186 	certain time and are not in a specified list of exceptions.  Since
    187 	'nightly' timestamps the start of the build, and findunref uses its
    188 	timestamp (by default), this can be used to find all files that were
    189 	unreferenced during a nightly build).  Since some files are only used
    190 	during a SPARC or Intel build, 'findunref' needs to be run on
    191 	workspaces from both architectures and the results need to be merged.
    192 	For instance, if $INTELSRC and $SPARCSRC are set to the usr/src
    193 	directories of your Intel and SPARC nightly workspaces, then you
    194 	can merge the results like so:
    195 
    196 	$ findunref $INTELSRC $INTELSRC/tools/findunref/exception_list | \
    197 	  sort > ~/unref-i386.out
    198 	$ findunref $SPARCSRC $SPARCSRC/tools/findunref/exception_list | \
    199 	  sort > ~/unref-sparc.out
    200 	$ comm -12 ~/unref-i386.out ~/unref-sparc.out > ~/unref.out
    201 
    202 hdrchk
    203 	checks headers for compliance with OS/Net standards (form, includes,
    204 	C++ guards).
    205 
    206 hgsetup
    207 	creates a basic Mercurial configuration for the user.
    208 
    209 hg-active
    210 	helper used by webrev to generate file lists for Mercurial
    211 	workspaces.
    212 
    213 install.bin
    214 	binary version of /usr/sbin/install. Used to be vastly faster
    215 	(since /usr/sbin/install is a shell script), but may only be a bit
    216 	faster now. One speedup includes avoiding the name service for the
    217 	well-known, never-changing password entries like 'root' and 'sys.'
    218 
    219 interface_check
    220 	detects and reports invalid versioning in ELF objects.
    221 	Optionally generates an interface description file for
    222 	the workspace.
    223 
    224 interface_cmp
    225 	Compares two interface description files, as produced by
    226 	interface_check, and flags invalid deviations in ELF object
    227 	versioning between them. interface_cmp can be used between Solaris
    228 	gates to ensure that older releases remain compatible with the
    229 	development gate. It can also be used to validate new changes to
    230 	the development gate before they are integrated.
    231 
    232 lintdump
    233 	dumps the contents of one or more lint libraries; see lintdump(1)
    234 
    235 keywords
    236 	checks files for proper SCCS keywords.
    237 
    238 makebfu
    239 	simple wrapper around 'mkbfu' for use outside nightly (when in a build
    240 	shell from 'ws' or 'bldenv'). 
    241 
    242 mkbfu
    243 	makes cpio archives out of the proto area suitable for bfu'ing.
    244 	Used by 'nightly' and 'makebfu'.
    245 
    246 ndrgen
    247 	Network Data Language (NDL) RPC protocol compiler to support DCE
    248 	RPC/MSRPC and SMB/CIFS.  ndrgen takes an input protocol definition
    249 	file (say, proto.ndl) and generates an output C source file
    250 	(proto_ndr.c) containing the Network Data Representation (NDR)
    251 	marshalling routines to implement the RPC protocol.
    252 
    253 nightly
    254 	nightly build script. Takes an environment (or 'env') file describing
    255 	such things as the workspace, the parent, and what to build. See
    256 	env/developer and env/gatekeeper for sample, hopefully well-commented
    257 	env files.
    258 
    259 pmodes
    260 	enforces proper file ownership and permissions in pkgmap and package
    261 	prototype* files.  converts files if necessary
    262 
    263 protocmp
    264 	compares proto lists and the package definitions. Used by nightly
    265 	to determine if the proto area matches the packages, and to detect
    266 	differences between a childs proto area and a parents.
    267 
    268 protocmp.terse
    269 	transforms the output of protocmp into something a bit more friendly
    270 
    271 protolist
    272 	create a list of what's in the proto area, to feed to protocmp.
    273 
    274 rtichk
    275 	checks that a set of CRs have approved RTIs.  Primarily used
    276 	by wx
    277 
    278 sccscp
    279 	copy a file under SCCS control to another location in a workspace.
    280 	also updates teamware's nametable.
    281 
    282 sccshist
    283 	Display the history, comments and diffs, of a file under SCCS
    284 	control.
    285 
    286 sccsmv
    287 	rename a file under SCCS control to another location in a workspace.
    288 	also updates teamware's nametable.
    289 
    290 sccsrm
    291 	delete a file under SCCS control workspace. also updates teamware's
    292 	nametable. Actually renames it to .del-<file>-`date` so that others
    293 	will see it move when it is brought over (in case they were working
    294 	on it).
    295 
    296 ws
    297 	creates a shell with the environment set up to build in the given
    298 	workspace. Used mostly for non-full-build workspaces, so it sets up
    299 	to pull headers and libraries from the proto area of the parent if
    300 	they aren't in the childs proto area.
    301 
    302 wx
    303 	A great workspace tool by bonwick. See wx.README for information
    304 	and warnings.
    305 
    306 wx2hg
    307 	Converts a TeamWare workspace under the control of wx to a
    308 	Mercurial workspace, discarding intermediate deltas.
    309 
    310 tokenize
    311 	Used to build the sun4u boot block.
    312 
    313 webrev
    314 	Generates a set of HTML pages that show side-by-side diffs of
    315 	changes in your workspace, for easy communication of code
    316 	review materials.  Can automagically find edited files or use a
    317 	manually-generated list; knows how to use wx's active file for
    318 	lists of checked-out files and proposed SCCS comments.
    319 
    320 which_scm
    321 	Reports the current Source Code Management (SCM) system in use
    322 	and the top-level directory of the workspace.
    323 
    324 wsdiff
    325 	Detect object differences between two ON proto areas. Used by
    326 	nightly(1) to determine what changed between two builds. Handy
    327 	for identifying the set of built objects impacted by a given
    328 	source change. This information is needed for patch construction.
    329 
    330 
    331 How to do a full build
    332 ----------------------
    333 
    334 1. Find an environment file that might do what you want to do. If you're just
    335    a developer wanting to do a full build in a child of the gate, copy the
    336    'developer' environment file to a new name (private to you and/or the
    337    work being done in this workspace, to avoid collisions with others). Then
    338    edit the file and tailor it to your workspace. Remember that this file
    339    is a shell script, so it can do more than set environment variables.
    340 
    341 2. Login as 'gk' (or root, but your PATH and .make.machines for dmake will
    342    not be right). Run 'nightly' and give it your environment file as an
    343    option. 'nightly' will first look for your environment file in
    344    /opt/onbld/env, and if it's not there then it will look for it as an
    345    absolute or relative path. Some people put their environment files in
    346    their workspace to keep them close.
    347 
    348 3. When 'nightly' is complete, it will send a summary of what happened to
    349    $MAILTO. Usually, the less info in the mail the better. If you have failures,
    350    you can go look at the full log of what happened, generally in
    351    $CODEMGR_WS/log/log.<date>/nightly.log (the mail_msg it sent and the proto
    352    list are there too). You can also find the individual build logs, like
    353    'make clobber' and 'make install' output in $SRC, under names like
    354    clobber-${MACH}.out and install-${MACH}.out (for a DEBUG build). These
    355    will be smaller than nightly.log, and maybe more searchable.
    356 
    357 Files you have to update to add a tool
    358 --------------------------------------
    359 
    360 1.  Add the tool in its appropriate place.
    361 2.  Update the Makefile as required.
    362 3.  Update usr/src/tools/SUNWonbld/prototype_*.
    363 4.  Update usr/src/tools/README.tools (this file).
    364 5.  Repeat 1-4 for any man pages.
    365