Thursday, January 17, 2008

Game Installers -- picking a directory

Over in Raymond Chen's blog I wrote a short comment about game installers and letting the user pick their installation directory. You can read it now at The Old New Thing and look for 'Peter'.

Raymond is right: your computer is already making hundreds of decisions for you, and nobody says word one. Then it makes one other decision that has just as much impact on you, and you get upset. That's silly. He's also right that exposing these decisions to the user is expensive: it wastes the time of people who don't care, it sucks up QA resources, and when people set it wrong (and they will) they call and complain. Or they don't call and uninstall my company's product, and I don't like it when that happens.

To put the problem in perspective: do you argue with the computer about the order in which USB hubs are enumerated? Do you ever find yourselve wishing you could load your program into one of your memory SIMs and not the other? Why can't it go in the other direction? Who picked "HKEY_CLASSES_ROOT\CAPICOM.Store.3" as a registry entry anyway? Why can't you pick that, too? And the answer is: because someone picked a way that's as good as any other way, and you just have to live with it.

Let me describe the actual program I work on. It's a sort of program manager that lets you to pick some programs, and the manager application downloads and installs them. One of our big advantages over the "classic" way of doing things is that we try to have a very smooth download and install experience with no pointless pop-up messages. I recently had the mispleasure of installing the same three games (that my company doesn't control) on a half a dozen laptops. In each case there was just dialog after dialog -- please pick a directory to install to, HEY! THAT DIRECTORY DOESN'T EXIST! OMFG, THE WORLD WILL EXPLODE! DID YOU REALLY MEAN THAT? -- would you like the program to actually work? -- would you like us to spam you? -- would you please read a this long text and mindlessly click OK? -- you have to scroll to the bottom first!. Yuck, yuck, yuck.

So what about the comments people left? Well, my company is picky about what programs are offered, and only allow programs that work when installed to %PROGRAMFILES% and in standard user mode, so Alexey's comments don't apply. We also handle the namespace issues that Will raised. I'm dubious about the namespace collision issue anyway -- the registry would have the same issue, as might the class names, along with the 'universally writable data' directory in c:\ProgramData. As far as disk space issues go (like SuperKoko and Igor Levicki raised) -- well, the programs aren't very big. cjm thought I had a bad attitude, but I don't. Indeed, my bosses often appreciate my general willingness to work on what needs to be worked on. cjm also denies that it causes more work, but that's not true and I have the bug reports to prove it.


Evan said...

I install everything to a different drive (G:\Program Files\... if you're really curious) despite my %PROGRAMFILES% remaining on C:, and I get very annoyed when programs insist on putting themselves on C:.

Why do I do this?

I don't want programs installed on my system drive, for a variety of reasons. But at the same time, I don't know what all is involved in changing the program files directory. I know that you can edit the registry or (based on a search) use TweakUI, but will things continue to work after? I don't know. There's stuff in C:\Program Files off of a clean Windows installation. Will they continue to work, or is there something more that you have to do? I don't know.

I could try it, but at the same time, I don't feel like sorting out mysterious failures if it turns out that you have to do more. So I just change the directory when running the installer.

As for space, how big is "not very big"? If you're talking 10 or 20 MB, then I would probably just roll my eyes. If you're talking 100 MB or more, I would start to get pissed. My system partition is small, and 100s of megs start to add up over the course of a couple years.

Triangle said...

"Raymond is right: your computer is already making hundreds of decisions for you, and nobody says word one."

I don't think this is the intent of Mr. Chens' post. He is remarking about where "dangerous" settings should be, not how fine grained those settings should be.

To put the problem in perspective: do you argue with the computer about the order in which USB hubs are enumerated? Do you ever find yourselve wishing you could load your program into one of your memory SIMs and not the other?"

There is a very big difference between these examples and the filesystem: The filesystem is highly visible to the end user. Partitions and drives have limited amounts of storage, and it isn't uncommon for people to rearrange their filesystems differently from the Windows defaults. It depends entirely on the user: Some people are very picky (Sometimes obsessive) about how their files are organized, and have elaborate trees and naming systems. Some people (Like me :[ ) throw everything in the same directory so that you can access anything with at most 2-3 clicks. Some people don't care. Regardless, the filesystem is an important part of the system. Failing to work correctly with it will cause people frustration. Don't forget the customer is always right. That doesn't mean you have to implement everything the customer asks down to the smallest detail. But you do have to implement as much as possible within reason.

Dean Harding said...

Your comparisons are not valid. The registry has totally uniform performance and capacity. It actually doesn't matter what order USB hubs are enumerated in, and memory is obviously just a flat address space -- the performance of the memory at location 0x17F6AB is the same as the performance of memory at location 0x7A65FBC.

Disks are different. Two disks could have very different performance characteristics. I could buy a cheap 20GB or my C:\ drive and a nice fast 400GB deal for my D:\ drive.

You say your programs are "small" but what I've only got 100MB free on C:\ drive. Why would I install anything there when there's 200GB free on D:?

There are many reasons why I might choose to install on D:\ instead of C:\. There are NO reasons for me to choose a different registry key, USB enumeration-order or (I can't believe I'm even writing this) memory location.

You say you want to reduce the number of dialogs the user has to click through to install your applications. That's fine, but then why not make it an option of the manager, so you can set it once and forget it?

And finally, if a program is simple enough that I shouldn't have to choose the install location, then it should be simple enough to not require an installer at all.

SuperKoko said...

To put the problem in perspective: do you argue with the computer about the order in which USB hubs are enumerated?
No, but I badly need a way to map a USB mass storage drive to a specific drive letter or to a specific directory.

I'm dubious about the namespace collision issue anyway -- the registry would have the same issue, as might the class names, along with the 'universally writable data' directory in c:\ProgramData.
Two different versions of the same program have the same name in 99% of the time.
There're serious versionning issues related to programs using the registry.
I intend to develop a program that would virtualize the registry base as seen by every program so that I can install two versions of the same crap software at the same time.
(I'll handle sharing of registry keys with a hierarchy of shared settings between programs)
The other issue with the registry is that applications stupidly think that they can change their settings format at every minor release, which make impossible to use roamable profiles on a not-perfectly-uniform environment.
The main advantage of *NIX text configuration files compared to Windows' registry base isn't due to the model itself, but due to what applications DO. Windows applications don't understand that people may want to migrate their settings. Consequently, it's impossible to reuse settings, unlike *NIX config files which are designed to be seen by the user, and thus, don't evolve too fast.
Anyway, that's off-topic.

I guess I'll have to do the same thing for the file system.
We'll come up to a day where every program will have to be run in a different VM environment.

As far as disk space issues go (like SuperKoko and Igor Levicki raised) -- well, the programs aren't very big.
But my 20GiB disk is full. Games easily take 2 or 4GiB each.
And, yeah, I want to use Windows 98 SE on my old computer (hence, I don't have junction points on it), because, whatever people say, it's MUCH faster than Windows XP on this computer.
I guess, my non-standard choice mustn't be supported by modern software. They don't care if their program cannot install on the previous version of Windows, as they don't care if it cannot be installed on the next version of Windows.

Actually, my old computer has a 2GiB partition for the system so that I can fully backup it very quickly with a raw block-level copy.

I must also say that I like very much applications providing a way to set every path such as the Opera browser.
What can I do with the great Opera software?
I can store it on an USB key, with all its user settings, and execute from it anywhere, and it works!
By default, the cache directory is stored in "Application Data" which has serious performances issues on my university computers where I've set it to point to a network drive (Z:\UserProfile\Application Data).
Fortunately, this can be changed! So I changed it to "Local Settings\Application Data".
Hey, I could even install Opera with non-admin rights because I had the choice of disk! I installed it in Z:\Opera854.
Without this, I could never have installed Opera.

I also have several different differently-tuned versions of Opera on my computer.
I have two different versions (Opera 8 and Opera 9) sharing many of their settings! This wouldn't have been possible if dirs weren't configurable. Both browsers have their advantages, and depending on the sites I visit, I use one or the other.
I have other clean "test" versions with clean profiles.
I have Opera 8.54 installs. One for tests, and one for everyday use.
The /settings option also allows me to use different profiles with the same Opera install.

Without this wonderful directory customizability, I wouldn't use Opera.

Sunrise Programmer said...

Cool -- people care enough about this to leave comments!

evan: I feel your pain. I also used to have a %PROGRAMFILES% on a D: drive, and it didn't work very well -- in particular, Office insisted on putting hundred of megabytes on the C: drive no matter what you specified as the install drive. Yuck, yuck, yuck.

In my case, yes, the programs are pretty small.

triangle: Raymond also points out that some choices shouldn't be exposed at all. And in previous entries he's said that each choice that you make someone take is expensive even for people who pick the default.

The file system is a very interesting case: some stuff is "exposed" and others are not. Early computer users actually complained about file systems: they instead just wanted raw access to the tracks and cylinders (although these were drum tracks, not disk).

Lots of stuff isn't exposed, or is exposed (IMHO) in an irritating way. Like how "person's Documents" is renamed to "My Documents" if you are logged in as the right person.

My personal fav. for filesystems (even though I haven't used it) is the Sun "Thumper". Every year you open up a (heavy) drawer, pull all the disk drives with a red light, drop in some new ones, and shut the drawer. The system handles the old drives being pulled, and the new ones added, and will format and start to use them right away. That's how it should be.

Dean: Actually, all of these things have variable costs. And the key to the registry is: why do people want to have their own naming scheme for directories, but not registry keys? And memory sure does have different characteristics: some is faster, some is slower. And some memory locations are faster, too (or at least they used to be -- IIRC there was an "issue" with the 486 in the 1990's such that memory below 512Meg was faster than memory higher than that. Or so my ancient memory recalls).

As far as putting the dialog in the manager -- well, that's what we did. I never said we didn't implement the choice -- I just said that we didn't want to, and that it has very real costs.

The programs have to have installers. They're not so simple that we can just dump the files on disk.

Superkoko: May I suggest just running virtual PC sessions to get different registries? It'll get you running much faster than trying to make a program to do it, and is less likely to have god-awful horrid issues.

I've had luck migrating settings by just exporting and importing bits of the registry. Still, you're right that applications don't like to "move" very much. It's a problem that I'm surprised that Microsoft hasn't worked on more-- for me, it's the number one reason I don't update my computer more often.

Dean Harding said...

There is no cost difference between different memory modules. If you plug a faster module in with slower modules, they will all run at the slower speed.

(Of course, that's only true of single-CPU and SMP multi-CPU setups, but CMP and non-symmetric multi-processor machines are so uncommon and special-purpose as to be irrelevent here. You certainly would never run a *game* on one of them).

"why do people want to have their own naming scheme for directories, but not registry keys?"

Two reasons:

1. Most people never even see the registry. That's a programmatic interface only (mostly), and
2. The is no performance or capacity difference for different locations in the registry.

You seem to dispute #2, do you have any specific examples? The registry is all in a single file (well, one file per hive) so how COULD there be a difference?

Of course allowing the user to customize the path has a cost. ALL features have a cost to implement, you wouldn't say "we're not going to you allow to resize or move the application window because of the cost of implementing the WM_SIZE and WM_MOVE messages", would you?

Evan said...

"in particular, Office insisted on putting hundred of megabytes on the C: drive no matter what you specified as the install drive. Yuck, yuck, yuck."

Visual Studio does this too. It's aggravating.

Igor Levicki said...

We seem to have several issues here... lets deal with them one at a time.

10 User is always right
20 GOTO 10

Explanation: You cannot argue with a User. User _owns_ the hardware and has paid for software.

Analogy is very simple -- User's computer is their HOUSE, they are the HOST, and your software is a GUEST.

Guests should BEHAVE in whatever manner their host expects them to behave. If they misbehave, they get thrown out. It is that simple.

Users want CONTROL. Well, at least those amongst us who are not yet lobotomized. I hate it when software starts bossing me around: "Click here to start", "Disk space is low, click this icon to run disk cleanup", "You must restart your computer now", "You have just updated your version of Crappy software's BloatKing to 33.1.65534 build 1 beta2, click here to see what's new", "your antivirus definitions are out of date click here to blah, blah", etc...

Filesystem is a sacred ground. Applications litter with their files in way too many places. I have a problem with that. I want them self-contained and movable. I want them to have settings in a plain-text file which I can edit and transfer to another computer. I do not want stinking 2MB installer for 200KB executable.

Hell, I have even gone so far as to CRACK the NT kernel and shell to prevent them from creating those dreaded Recycler/Recycle Bin and System Volume Information folders. I decide what gets created in the root of my drive.

I also hate it when a program ties itself to installer somehow (PerfectDisk comes to mind). It installs no less than four frakking services and if you stop them they get restarted immediately, and if you delete their executables to prevent them from starting they get reinstalled immediately by the installer and if you have deleted the installation source it gets stuck using 100% resources so you can only reboot and then it starts immediately again in a vain attempt to repair the application -- that behavior (systematically supported by Windows Installer) borders with malware.

As for installing games -- for those multi-GB monsters, the best way is to put everything in a virtual FS (aka one big file) and just copy that file from DVD to the HDD. Not only installation will be faster but also performance will be much better because one big file does not need a lot of seeking like thousands of small files do.

There is more to all this but I am just too lazy to type more at the moment.

marius said...

I have two hard drives but when I've bought the first drive (200GB), I've partitioned it so that C: has 8 GB, D: and E: have 90GB.

The basic concept was that C: is for operating system, D: is for source code, MySQL server, databases and video processing, E: is for games, movies and music.

I thought 8GB would be enough for Windows but then Visual Studio forces itself onto C:, .Net Framework installs itself into Windows and already reaches 200MB and eventually you get bored to change the path for each application each time you install one.
So now, I have 500MB of free space left on C.

It doesn't make sense to have games installed in C:\Program Files, considering the actual "program" part of a 4-8GB game is about 10MB and the rest are game resources (textures, sprites, sounds, movies).
It's also stupid in some cases to assume the C: drive is large.

It wasn't really nice when I tried to install The Longest Journey and I couldn't, because it assumed I want to install it in Program Files but obviously, there were no 7GB free space.

See here the comic of the situation and also see the reactions of the people.

GreaseMonkey said...

Here's the common process:
- Welcome to Blah v1.2.
- EULA, if needed. Accept/reject.
- Select a directory to install to.
- Start menu folder: user has option to not create a start-menu folder for whatever reason.
- Other options: Desktop icon, Start menu icon, Toolbar icon, Run on startup, you know the drill.
- *** Install. ***
- Final options: Run program, read readme/changelog.
- If you restart, it *ABSOLUTELY MUST BE OPTIONAL*. If it isn't, I will personally saw off your reproductive organs.
- Finished!

Follow it. I would rather get a crapload of dialogues than have it try and control *MY* computer.

Heck, make it an option to have it all automated or completely manual, if you want to go for the "Automated" approach.

Improfane said...

I'm one who wants to choose where to install software.

I follow up your post in my blog.

Sunrise Programmer said...

[Here I am again]

So there are now more comments -- and yet, they aren't (mostly) more informative than previous ones. Sorry everyone, but that's how I see it. Some of you have decided to manually manage your disk space by making partitions. Everyone who's done so seems to think it's a good idea even though (a) it's causing you pain and (b) nobody has mentioned as single time when it's been useful.

There are some comments about name space collisions. My view is: name space collisions are inevitable. If you don't have issues with the registy, you get issues with your disk drive. Or your database (Google Gears is setting us up for this). Or in your Java namespaces. Or somewhere. The most complete and least hassle free solution now is to just run different versions on seperate virtual PCs. Any other "solution" is just a stopgap.

Other people have misconfigured their systems by getting a new hard drive and instead of getting a much larger one and copying everything over, decided to try to split their workload over both disks. As they've found out, Windows doesn't really "do" that. (And yes, I do this at home. I at least put all of MyDocuments into the D: drive -- that configuration seems to work OK).

My earlier comments on memory were, as near as I can tell, no longer valid. It's the penalty for programming so long: things that were "common knowledge" once aren't true any more.

Try reading any book from the 90's on networking, for example. There will always be an obligatory comment about setting your IP (which, thanks to DNS we mostly don't do any more) and your subnet mask (obsoleted by pervasive NAT).

It used to be the case (unless my memory is totally hosed) that the memory modules themselves all had the same "address speed" -- that is, they could all pick up addresses from their address bus at the same speed -- but had different "return results" speeds. There would be different number of bus cycles that different memories would respond with; they would set a "data good" bit when they had come up with the data.

Silly me. That's not how current memory works. Current memory can't handle multiple-speed memory.

I totally agree about restarts. I hate them. I especially hate programs which reboot just to install a new installer. Way to mess things up, Microsoft! In an earlier life I was installing a bunch of graphics programs to test for compatibility (with the CGM standard). Grr -- one of them rebooted a server with no notice. But that was a while ago; I haven't noticed installers doing that.

Greasemonkey: your list of things to do on install is, IMHO, much too long. Welcome screen? Eula? Nonsense; nobody cares. Not even lawyers (EULAs are of very dubious actual use, and mostly state things that would be true even if you didn't present them). and a "README" -- that's old fashioned now. Most people won't read them, so why include them? Apart from making the programmers feel smug ('the users should have known that file->exit would delete all their data. It's in the readme') it's really a bad solution to a bug.

Igor Levicki said...

You seem to have serious logic problems.

>>(a) it's causing you pain

Pain? How? By making it easier to find what I am looking for? Do you throw all your stuff on a heap in your house and then dig through it?

>>(b) nobody has mentioned as single time when it's been useful

1. If you accidentaly delete important file(s) what is the chance of recovering them if they are on the same partition as Windows swap file and registry hives?

2. How many times your system had to chkdsk partition D: after power loss? And partition C:?

>>As they've found out, Windows doesn't really "do" that

I don't understand what exactly you are talking about.

>>which, thanks to DNS we mostly don't do any more

Not true, many people do because they use private IP and NAT/port forwarding on their routers.

Evan said...

"(b) nobody has mentioned as single time when it's been useful."

How 'bout the /two/ times when my C: got corrupted? (One was, I think, a HDD failure. The other was software.) Because I had stuff on the other drives, I only lost a very small amount of important stuff.

This argument isn't *terribly* convincing because you really need to reinstall programs after reinstalling Windows anyway (that's another discussion), but there was still a number of programs installed to D:\Program Files that I was glad survived.

"As they've found out, Windows doesn't really "do" that."

And most of the reason Windows doesn't really do that is because of programs that don't ask you where it should be installed, or don't obey you.

Some is Windows design though.

Igor Levicki said...

Talk about hardcoded install paths:

Sunrise Programmer said...

Hah! My memory was correct! From The Indispensable PC Hardware Book, third edition, page 144: "The number of wait cycles can also depend on the location of the memory". It seems that there was faster and slower memory, and each memory unit did its own wait cycle management. Among other things, reading and writing could take different amounts of time.

The book, BTW, is from 1997.

ShadowWolf said...

Sorry, but I don't think you could be any more wrong.

Firstly, from experience in actually developing installers I can say that the vast majority of lawyers DO want the EULA displayed and specifically accepted by the user. Many developers, in fact, want to make sure that the user has scrolled down through the license to the bottom. Why? I don't know, but it's irrelevant. I'm not a lawyer. This is what people want to do with installations.

As far as I'm aware, Wait Cycles were more a legacy thing for before pre-charge and other tech came around.

But, the fact remains that at no point in time does changing something from %ProgramFiles% to D:\ cause any issue and it can resolve many headaches. Ie, what if your game requires a lot of modding? %ProgramFiles% on Vista has no user permissions, but I can more easily just set D:\ as full read/write for Everyone and install normal applications to %ProgramFiles% and things i plan on putzing with a lot to D:\ could I not?

Just 'cuz you don't want to use environment variables and non-system specific paths doesn't mean that the rest of us should suffer in my opinion.

Further, the registry is irrelevant because it's supposed to be a centralized data store to allow for more dynamic use of data - eg, Installing to a non-standard directory.

So, for example, you're supposed to have HKLM\SOFTWARE\MyCompany\InstallPath so that you can make InstallPath arbitrary.

Case-in-point is Blizzard has been doing this for years on end and hasn't had issues with aribtrary installation paths and they're only sitting at 10 million subscribers for their current flagship product.

Sunrise Programmer said...

Shadowwolf: thanks for you comments.

To jump right in: EULAs are in fact for the company I work for a non-issue. You might have guessed from my comments that I work for a company that has a game-downloader that works with different developers games -- and they are always willing to be covered by the one EULA that we displayed when the user first used our downloader.

Yes, wait cycles are now in today's dust bin. I had to do to a ten-year old book to find out the information. However, when I first started working with PCs, the idea of variables wait cycles was still in the future.

Trust me, moving the installers around causes issues. Imagine you're in firefox, and you are downloading some files -- and right in the middle you decide to switch the download directory. And then you reboot. When firefox starts up again, does it restart in the same directory?

More importantly, how much QA time (which is serious money) do you spend validating that everything will just work?

Bleah. Too much is the answer. What if the user puts the download directory onto a thumb drive? What if the user pulls it? What if they put a new one int?

Bleah, bleah, bleah. It's work to come up with the scenarios, it's work to decide which ones are important, and it's work to decide how to QA it. Remember that QA is the gift that keeps on giving: every release we have to test this functionality.

Re: VISTA. Again, it's a "I've configured my system wrong" issue -- in particular, you've got a game that doesn't work on Vista, but you want it to run.

The registry is relevant because one of the earlier comments included versioning issues -- they want to install two different copies and have them not interfere. However, since almost all programs assume full control over a non-versioned and unchangable registry location, that dream is generally impossible. Unless you use Virtual PC which, IMHO, is a godsend.

Yes, Steam does it. How do you know I don't work there? I have specifically mentioned that my company's product does support changing the download directory; I just happen to think that people who want to change the directory are silly.

No offense. But that's how I see it. And darn few of the comments so far are even close to making me want to change my mind.

And igor: you say, "the customer is always right". Nope. I make things that people generally want (and I can prove it with our download statistics). I'm not making it for everyone. Lots of people really, really like to download and install programs without having to click a single extra button.

Igor Levicki said...

"And igor: you say, "the customer is always right". Nope. I make things that people generally want (and I can prove it with our download statistics)."

Funny how you chose the must trivial thing I said for a reply... never mind, I can still beat your logic -- if you claim that the customer is _not_ always right, then what makes those customers who want to have less options right over those who want to have more options?

Those who want less options can simply ignore the extra options -- the rest of us can't add them if they aren't there. Your logic both as a human and as a programmer is flawed.

"Lots of people really, really like to download and install programs without having to click a single extra button."

Yeah right, next thing they'll ask is for you to play the game for them, and remember -- you are the one asking for it by siding with those lazy bastards.