Viewing comments for: New Tool!!!

News Downloads GameDev Links Tutorials Articles Forums Glossary Gallery IRC Chat Dev Blogs

Welcome Guest ( Log In | Register )

Project Wish > Blog list > njpaul's blog > Viewing comments for: New Tool!!!

New Tool!!!

Not rated (0 votes) PW Development · May 5 2008, 04:26 PM
Anyone who was on IRC this past weekend knows that I was looking for a solution to building our software on Linux. Makefiles are fine, but can easily get out of hand. I tried SCons, which was too verbose and supposedly doesn't scale down well. I tried CMake, but the lack of proper tutorials, and the lack of custom configurations (even though they say you can add new configurations), was frustrating. I tried boost build, but it was not able to fit in with our directory structure, unless I missed the docs on it, or it's undocumented. Finally last I settled for premake, which does 90% of what I need it to. The remaining 10% is fairly critical to this project, but I was able to work around it, and supposedly the features I need will be added eventually. Unfortunately I was not able to get the PWToolBox Python configuration built last night. Ultimately, nothing out there was able to give me the flexibility and scalability that I needed. So this weekend was an exercise in failure. When I got into work this morning I remembered that I had nothing to do. Luckily I had nothing to do all day. I was able to write a program that I like to call Mage, which stands for Makefile Generator. First off, Mage is not meant to be a cross-platform build tool (there are exceptions, see below). It will not generate project files for Visual Studio or Code::Blocks. It only generates Makefiles. Mage is essentially just a thin layer above make that makes it easy to customize your build process. It does not attempt to figure out settings for you based on configuration types. If you want all your projects to share common compiler flags (for example, debug and release flags), then simply write a Python script that stores these flags and import them in every build script. Alternatively, you could write a wrapper around Mage that handles those settings for you. So what features does Mage have? Builds a Makefile (duh). It does not compile your program. After generating the Makefile you simply run Make to do that. Build scripts are plain old Python programs. No need to install anything else. No need to run a separate tool over the script. You have the full power of Python to do your bidding. Just type "python <name_of_script>" and a makefile is generated for you. Mage is written in Python, with no supporting libraries. Just install it as a python package and you're ready to use it. The command line is completely accessable through the build script. Anything you can do with Makefiles you can do with Mage. Mage will detect dependencies between files. This eliminates the tediousness of Makefiles and allows for minimal rebuilds. Specify program type. You tell Mage what type of program you are building (executable, static library, or shared library) and it will fill in the correct compiler and linker flags. You could also specify the value None and control those flags yourself. As of now, the exe works fine, the shared library builds, but I haven't tested it, and I haven't implemented the static library. Uses the Visual Studio style Solution, Project, Configuration setup. This produces a target called <Solution>, a target for each project in the solution called <Solution>_<Project>, and a target for each configuration of each project in the solution called <Solution>_<Project>_<Configuration>. This allows you to make the entire Solution, make on a per-project basis, or make on a per-configuration basis. Supports aliasing targets. Don't like the names that are generated for you? Want the target name PWToolBox_PWToolBox_Debug to be renamed to PWToolBox_d? Yup, you can do it. (Well actually right now you can't, but the important thing is that you will be able to do it). Customizable directory structure. Supports placement of binary files and object files into specified directories on a per-configuration basis. Exclude files from a configuration. This is handy when working with SWIG. When you don't want to build a wrapper just exclude the file from the configuration. Add variables to the makefile configuration. If you want the user to specify the boost includes directory, just add a variable and give it a default. The Makefile will use that variable to determine the path. (This feature isn't implemented yet) Pre and Post build scripts. Pass a list of command lines that you want to run at certain stages of the build. (This isn't implemented yet) Readable Makefiles. Maybe you want Mage to generate a Makefile for you, but afterwards you'd rather handle it yourself. The generated makefiles are very readable, and include comments outlining the solution, project, and configuration sections of the makefile. It's very easy to find out what part of the Makefile is building what configuration just by glancing at the file. Portable to Windows. If you have Cygwin installed you can run Mage and generate makefiles. You can then run make as normal. Thanks to Cygwin, you don't even have to be in a Cygwin shell to do it. This is actually how Mage was created. You can use Python to help get around platform-specific issues. Not bad for 8 hours of work. I'll probably have this up in SVN on Wednesday (that's the soonest I can get access to SVN again), and we'll probably release it with PWToolBox. Don't know the license yet. I was thinking of going GPL with this one though.

Comments for: New Tool!!!

njpaul	May 6 2008, 03:26 PM
PW Client Team Lead Group: PW Team Leader Posts: 202 Joined: 27-June 06 Member No.: 611	I had another 8 hours today to work on Mage. Here's some updates: Parameterized the call to the compiler. Now if you want to specify gcc instead of g++, you can. Updated the dependency checking. Now if A depends on B, and B depends on C, A depends on C. Just for fun I wrote this part iteratively instead of recursively. Implemented aliased targets. Implemented support for variables in the Makefile. Improved the readability of the Makefiles. Improved the readability of the output of the Makefiles. The Makefile will make a very obvious comment as to what file you're compiling into what and when you are linking. It keeps the steps visually separated from each other so the output is easy to read. Turned it into a real installable Python package using Python's distutils. On Windows an installer is created. On Linux, an RPM is created for systems that support it, otherwise a tar.gz file can be used. I'd like to get deb packages working too, but Python doesn't support that out of the box. So I'm mostly done with it. I just have to finish the build type and the pre/post build scripts.


Jerky	May 6 2008, 05:13 PM
Former PW Project Manager Group: PW Admin Posts: 1,610 Joined: 11-January 05 Member No.: 62	LOL. I love work when there is no work.


Maxwell	May 10 2008, 04:36 PM
Master Group: Members Posts: 250 Joined: 11-October 05 Member No.: 560	Nice work nj, sounds like it was a fun project


Ethan	May 16 2008, 06:56 AM
Newbie Group: PW Developer Posts: 22 Joined: 11-August 07 Member No.: 1,101	Although I didn't quite get the cross platform issue, it seems indeed interesting. I have been messing with Code::Blocks to create automatic makefile. But like I said, it's a mess and as the feature depends on the IDE, that's really not a good solution. EDIT : QUOTE I was looking for a solution to building our software on Linux This implies that on Windows, there is already an automatic build process, isn't it? (Or an easy build through an IDE?)


njpaul	May 16 2008, 08:33 AM
PW Client Team Lead Group: PW Team Leader Posts: 202 Joined: 27-June 06 Member No.: 611	Yeah, on Windows it already builds through the IDE. I primarily develop on Windows, so the Windows build will always be available first, although this tool should cut down the time between having a Windows build and having a Linux build.