Local Links

External Links

Contact

Search this site

RB Project Tools - the lost pages

This page contains information about the original RB Project Tools.

Since then, these have evolved into Arbed - The Powerful Project Editor for Xojo.

There used to be extensive documentation on the rbdocs.com site, but the wiki site has vanished. You can find most of it here in the internet archive, still.

RB Project Vault Export / Import tool (RBVExIm)

Table of Contents

This page documents the RBPrjVaultExIm tool set originated by Thomas Tempelmann.

It is a tool to use common version control software such as CVS or, preferrably, SubVersion (svn) with REALbasic project files.

It is derived from the RB Project Tools

Current version

Current release version is 1.1.2

Go to the PrjTools web site to download it.

How it works

Basically, it takes your .rbp file and splits it up into separate files, one for each project item (module, class, window, interface, container control, and so on). Note that only project items are processed, but not text files, resource files, databases and other external items that are not source code related.

It puts all these files into a special folder next to your .rbp file. This folder is called the Vault.

The files in the Vault are text files (with their name ending in “.rbvx”). See below for details on this format.

These files in the Vault can then be passed to a Version Control system (SCM), such as SubVersion or CVS.

This tool is also able to recreate a project file from these files in a Vault, of course.

Hence, with this tool, you can manage your project with a version control system on a per-item basis and get a good view on changes. Much better than what you’d get if you would use RB’s XML project format instead. (Only disadvantage is that you need to invoke this tool as an extra step, but I’m also working to automate this part, at least if you are using SubVersion with the svn tool (or svnX) on OS X.)

Important note on usage

There is one very important rule to always follow:

Before fetching changes from the repository (check-out), you must make sure you have the latest changes in your vault.

That means: Before you use a version control client to see if there are updates available from the repository, you need to export your current project to the vault.

If you forget this, and if you had made changes to your local project, you would destroy your changes if you would check out the latest version from the repository and then import those changes from the vault to become your new project file. That happens because the vault import operation does not merge your project file with the vault, but simply replaces the project file entirely with the contents from the vault.

How to use

I recommend you get the Subversion software and, if you are on Mac OS X, use it with the easy-to-use svnX client (the svnX website points you to Martin Ott’s Subversion installer for Mac OS X, which saves you from searching the Subversion website).

Basically, after having created the initial setup, the workflow is always like this if you want to exchange changes between your local verison of the project and the (server sided) repository collecting all your changes in history:

  1. Perform a export to vault
  2. Use a svn client to see if you’ve made changes that need to be stored in the repository (check-in)
  3. Get the latest changes from the repository (check-out)
  4. Perform a import from vault

(more infos on setup to follow later, perhaps also a tutorial with svnX)

Things to consider

  • Non-code external items (text files, resource files, etc.) are currently not written to the vault. This may change, but until then you have to manage saving them to the repository separately.
  • If you have two separate working copies of your project then this will also usually mean that the path to any external items you may be using will be different from the perspective of the two copies of the main project file. Every time the IDE saves your project file, it will write an updated absolute path reference to these external items. This means that the two project files will always differ in these part entries (there is also a relative path reference which will usually remain identical in the copies). This means that when you check in any changes from Copy A and check them out to Copy B, then Copy B has the “wrong” path reference for its local system. So, once you open this project of Copy B and save it again, even without making any changes, it’ll have updated the paths to any externals it may reference, and this means that it’s now dirty and awaiting a commit to the repository. There are two ways to deal with this: (1) you are aware of this and know that this only affacts those small vault items that are external references. Those items usually have the ‘kind’ code “pExt” and contain just the name and the path reference of the real item. It’s not really necessary to ever commit these, until you change the item’s name or change its relative file location. Or (2) the rbxexim tool remove the “alis” and “path” records from the external item references so that only the relative “ppth” remains, which will usually not change.

The Text File Format (.rbvx)

A new format was chosen over the standard XML project format:

  • It is better suited for text comparison. This matters when you want to manually merge conflicting changes or just want to view differences between one version to the next.
  • The files are slightly optimized: Duplicate Super entries get removed (The RB IDE does often creates these duplicates and never cleans them up again, as of RB 2006r1) and other items get ordered so that even when the IDE rearranged some internal properties at random, the resulting text file is not showing a change - of course, nothing gets reordered that would affect the functionality of the project. (Admittedly, these optimizations could have also been applied to the XML format.)
  • The details of this format and its differences to the XML format used to be documented at the rbdocs.com wiki site. This site is not available any more, however, there is still a slightly outdated backup at the internet archive.

Why another project format - is that really necessary?

The .rbvx format just a different representation of the binary data - no information gets lost.

On the other hand, the XML format used by RS is prone to cause problems: For instance, in RB 2005r4 XML export files there were items with incorrect XML tags due to translation errors from the 4-letter codes used in the .rbp project file format, and backwards compatibility with RB 5.5 was not working well either.

The .rbvx format, however, is derived directly from the binary .rbp file format and can’t make these mistakes since it performs a reversible and verifyable translation but does not interpret anything.

So, as long as you want to use a textual format (which you need for version control), the .rbvx format is safer than the XML format.

Discussion Topics and Background Information

Conflicts between the project's original and the copy in the Vault

This topic explains a safety measure on the import functionality

When exporting a project to the Vault, it will replace all items in the Vault, meaning any previous data in the Vault gets lost. This is not a problem though, as the Vault is just a local copy of what is supposed to be stored in the repository, meaning you can always restore the Vault from the repository in case you’ve accidentally exported a project to the vault.

On the other hand, importing a project from the Vault is potentially more dangerous!

Consider you have two separate working copies of your project, e.g. you and a remote person working on the same project.

Now imagine this work flow scenario:

Both copies (persons) start out with an identical project. Then, both persons change their own copies at practically the same time. Now the first person checks in his modifications with the repository. The other person learns about this and checks these changes out.

So far, this is a totally normal process with a version control system.

It can become dangerous with the particularities of using the Vault, though:

If the second person does not export his own changes to the Vault before checking out the changes from the repository into the Vault, he will have two conflicting copies of the project: One in his .rbp file and one in the Vault.

Now, if the person performs the Import From Vault operation, it could effectively erase his own version of the project, because the Vault only contains the version checked into the repository by the other person.

(To resolve this conflict, the person has to export his own changes to the Vault and then merge the other person’s changes in the Vault.)

The danger that one can lose one’s own work if one forgets the Export operation still exists though.

As a safeguard, the Import From Vault operation attempts to detect if you have made changes to your project file since it was last exported to the Vault. It achieves this by storing the last modification date of the project file (and its external items as well) in the Vault. If, upon import, a file’s date does not match the stored dates, the import is stopped to preserve the changes made to the project.

Therefore, an extra file called “ExportState.localonly” is created in the Vault. It is important that this file is not committed to the repository! (A common solution to prevent this from accidentaly happening would be to make it invisible, but I am not sure yet if that causes new difficulties. Feedback on this is appreciated, preferrably on the REALsource list, see above)

Dealing with External RB Project Items

The Vault Ex/Import tool can handle external items. But if you’d then try to use them with a version control system such as SubVersion or CVS, it gets complicated easily.

Here are a few of the complications I see - may we eventually find a nice solution to deal with them:

Separate vault folders for externals are hard to manage for the user

If there are separate vault folders for external items, then the user has to manage each of them individually, which can be a nuisance. The user could also easily forget about checking all possible external vaults and hence miss updating the repository.

For now, simplicity demands that the user only deals with one “working copy” (that’s a Subversion term) per RB project. A solution to this could be that this tool manages the externals in a way that it joins them into the project’s single vault folder, or that I’ll provide a separate tool, similar to RS’ “RB Project Manager”, that manages shared items without reverting to the “External Items” kludge.

A separate shared items mananger might have the additional benefit of controlling for each individual project where it wants to update its shared items (so that as long as one does not update them, the project still works the same even if the new version of the shared item will have broken something with this particular project)

Creating a vault folder next to every external item

Currently, the code would create a “Externals.vault” folder for each external item it runs into (unless there exists already one). But if you have several external items nested in several folders, you’d also get nested vault folders. Example:

Original folder hierarchy of your externals:

MySharedItems/ 

   GlobalMethods.rbm
   Classes/
     Class1.rbo

After an “Export to Vault” it would like this:

MySharedItems/ 

   GlobalMethods.rbm
   GlobalMethods.vault/      <- one vault with .rbvx files
     GlobalMethods.rbvx
   Classes/
     Class1.rbo
     Class1.vault/           <- another (more hidden) vault with .rbvx files
       Class1.rbvx

So we end up with several .vault folders, part of them hidden deep inside out folders with out external items. This is not easy to manage using a Version Control system, I think. Instead, all these external items should be appearing in one .vault folder at the outside level, and inside it collects all items of the “MySharedItems” folder.

To achieve this, certain rules must be verified by the code: Since the user could have rearranged the vault folder hierarchy by adding externals on a higher level, a check must be made that the vault folder does not already contain other (older) vault folders. Also, each “.rbvx” file must contain information about how many folder levels upward the vault folder is to be found, and this must always be verified.

Thoughts about putting ALL project items into one vault per project

For the ease of use, it would be nice if all items used by a project, including External Items, would be placed into one vault. Then only this one vault needs to be exchanged with the version control database (repository).

This tool is easily able to do this, no question here.

The question is, however, what this means if users DO use External Items.

For one, it means that if two projects share the same item as an External Item, there will end up two separate copies of them in the repository. Which means that by looking at the repository, you cannot simply find the latest version of such an item any more, but would have to check every copy and see which one is the latest of them.

But once you would check out or update (i.e. retrieve from the repository) a project, its local version of the External Item would be retrieved and, through the post processing by this tool (rbxexim), would eventually overwrite the single, shared, External Item on your disk. Which is just fine for the project at hand, but might not be good if you’d then decide to open the other project without doing a updating checkout for this other project, because then this other project gets to use a different version of the shared file than what it had been using before the first project got updated from the repository.

This odd effect would require some understanding from the user. I wonder if that’s easy enough to grasp to follow this approach instead of having separate vault folders for the External Items.

Let’s try an example:

  1. Project A and Project B both use the External Item C.
  2. At the beginning, all 3 files are at version 1.
  3. Now let us update A, along with making changes to C. So A and C get to be version 2.
  4. This affects B as well, of course, and B might not compile any more because of the changes to C. This is just the normal way with External Items (and, after all you wanted to use them, that’s why we discuss it here).
  5. Since we’re finished with Project A, we now commit it to the repository (meaning items A and C are written to A’s vault folder and then both are written to the repository’s database)
  6. Now we want to work on Project B. Due to the changes to C, we assume in this example that we cannot build this project any more. So we decide to get back the version of C that we used before the changes. To do this, we check out the Project A(getting its set of files from the repository into B’s vault folder and then use the rbvexim tool to recreate that version’s files A and C).
  7. At this point, Project B can be built like before (without the changes made to C for Project A), but we must not build Project A any more at this point because it would mean that it would be using the older version of the C file now.
  8. So, unless you update both Projects A and B to work with the latest version of C, one must be careful to check out its version (including its version of C) before working on either project.

Does that all make sense? Do you think this can be a predictable way of working with External Items in order to have shared code between several projects?

Concerns

While the previous sounds like a good idea, it causes new complications in handling:

Imagine you have a project with external items stored in the repository and want to check out these files on a computer where these have not been stored before.

Checking out the files to the vault is no problem. But then the rbvexim tool should be able to recreate the .rbp and the external file(s) from it, right? But what if the external was originally stored in a totally different place, i.e. not next to the main prj file or in a subfolder of it, but on an outer level? How should rbvexim decide where to place the file, especially in cases where the relative path can not be recreated because this system has a totally different directory layout? Well, it would have to be told by the user where to place these externals.

But if the user chooses to store it in a different relative place than where it was in the original installation, then it means that the location of externals is not any more a property of the project file (as it usually is) but of the local system’s user. Meaning that rbvexim now has to start managing external items on each system?

A work-around would be to require that external items may not be in outer folders (relative to the location of the .rbp file) but only in folders and subfolders of the .rbp file. But that would be limiting.

REALbasic, Programming
Page last modified on 2011-03-05, 11:20 UTC (do)
Powered by PmWiki