Technological Devices, Focal Practices, and Running

This week I'm blogging at the Logic of Long Distance. Check me out there!

Database Version Control Goals

I'm currently version controlling a database and developing version control processes which will be used by a new team at my company. Everyone knows you should version control your database, but what exactly this means and how to go about doing it are not obvious. As I've mentioned before, version controlling databases is much different from version controlling application code, because you can never start from scratch with a database.

To meet the minimum requirements of a version control system (VCS), you must be able to:

1. Maintain a directory structure of all database objects
2. Maintain change/upgrade scripts
3. Isolate development from production code

The most basic version of Subversion will allow you accomplish these tasks, with two caveats. First, you have to either build your directory structure from the ground up, or you have to form some kind of process that will script out your database. I've tried the latter with Powershell, SSMS, Visual Studio database projects, and custom .Net code using SMO. In my opinion, the least bad option is .Net, since the code is not too difficult to write and you can customize it to your needs. You can add in functionality to script out only objects changed after a certain date. However SMO is incredibly slow. It took me 2 hours to create a baseline for my database.

The challenge shared by all of these home-brewed approaches for scripting out database objects is that you have to be consistent. For instance, if sometimes you use SSMS and sometimes Powershell, the object scripts may be slightly different and create false positives in your VCS.

If you stick with these minimum goals, you are also hampered by the annoying and uncertain process of having to maintain change scripts on top of your object scripts. I'm sure you always test your change scripts thoroughly on test environments that exactly mirror production, but you can never be 100% sure that your codebase actually matches your production database. Finally, if you want to create sandbox environments, you'll have to create another process for turning your object scripts into database generation scripts. This is more important if you work at a software company and need to be able to reproduce defects in old, supported code than if you work in an in-house database system. In either case, it's still a good idea to be able to create sandboxes for developers so that people don't break each other's code from working in the same database.

For these reasons, you also want to be able to:

4. Generate change scripts automatically
5. Create database environments with test data at any version
6. Know what version a database is

Now we're talking about incurring some monetary costs, but most likely ones that will pay for themselves. The two main options for achieving these advanced goals are Red Gate tools and Visual Studio Ultimate / Database Edition.

Visual Studio database projects are a cool idea, since you can populate object scripts and compare them against your database, as long as you have something better than VS Premium. You can get Anhk SVN or Visual SVN to version control directly in VS. VS has an 'offline' development model, meaning that you do your code in VS and then later create change scripts by 'compiling' your database. I've had issues with circular dependencies among databases, making this an unfeasible process. You get what you pay for with Red Gate tools, and you'll really want both Source Control and SQL Compare. Source Control just helps you deal with the basic goals in an elegant way, but, when paired with SQL Compare, it will allow you to create scripts to turn any database at any version and turn it into one of any other version. You can also get tools like Data Compare to sync your test and production environments, and you can use the Data Generator to create non-sensitive test data. These features improve the development process, but are not necessarily within the scope of version control.

There are a few options for knowing what version a database is on. SQL Compare will allow you to compare the database with a version in a repository. You could also create a table that stores the database version, which is useful but will never provide 100% certainty about all the objects. Finally, you can create extended events for database objects but Phil Factor doesn't recommend it in the eBook, since there are not many options for documentation. Database version control is slowly making it's way out of the Dark Ages and into the modern, Agile world that other developers have long taken for granted. Once you are able to handle all of the advanced goals, you can do even more things, such as automate the build process and run automated test scripts, a process called Continuous Integration. I must admit that the developments like those Red Gate has made take some of the mystery and the fun away from good ole database development. But I'll probably get over it.

Links:
-Grant Fritchey's article on Simple Talk, which is chapter 3 in the Red Gate eBook
-Michael Baylon's own list of goals and solutions
-Another good article

Some Questions Concerning Technology

After you work with computers for a while, you stop asking what acronyms stand for if you know what's good for you. The answer is always long and pointless. SQL is hardly a Structured Query Language. GNU is not exactly Not Unix. And Microsoft comes up with a new three-letter acronym (TLA) every week. Luckily, most companies have turned their IT departments into just Technology departments, so we have only one letter to worry about. Though it may be a pursuit both long and pointless, I've been thinking about what technology really is.

There are two standard ways of understanding technology. One is that it's the essentially human activity. Monkeys use sticks and bees build nests, but humans take tool-making to an unprecedented degree due to their clever brains, opposable thumbs, and upright posture. Various intellectual revolutions have taken humans farther and farther from their natural state.


Another way of understanding technology is as applied science. Wikipedia says technology is "the making, usage, and knowledge of tools, machines, techniques, crafts, systems or methods of organization in order to solve a problem or perform a specific function." While science can be pursued for its own sake and without a clear view of its potential application, technology is the use of knowledge for specified ends.

There are a few problems with these definitions. For example, if technology is just part of what we are, then why do people often rail against technology, as happened in the industrial revolution, or after Hiroshima, or in today's world of hyper-connectedness? If you say technology is "natural" or "human", then it's hard to explain why some technology is good and some technology is bad.

Similarly, if technology is just problem-solving, then why does it have so many unforeseen effects? It often seems that new technologies create as many problems as they solve. In The Social Network, Sean Parker says they can't monetize Facebook because they don't even know what it is yet. Technologies as simple as email have changed they way we live--they don't just scratch an itch.

In a characteristically gerund-filled essay, The Question Concerning Technology (1954), Martin Heidegger tries to overcome these obstacles and get at the essence of technology. First, he distinguishes between the production of technological artifacts and the way of relating to things that makes their production possible. It is only when humans treat things (e.g., the sun, information, and even other humans) as means to an end that we can create technological solutions. For example, when a river is understood as a means to an end, it can be dammed to produce power. It can be understood in other ways, (e.g., a thing of beauty, a shape, a manifestation of God, etc.), but such thinking doesn't help solve problems and is thus not technological. Heidegger argues that technology, or technological means-ends thinking, is what makes modern science possible, not the other way around.

Neither science nor technology are necessarily bad things, but it's easy to take means-ends thinking as the only appropriate way of relating to the world around us. Today, as Heidegger points out, it's hard to take seriously Aristotle's four causes (material, teleological, formal, and efficient), since we are so used to thinking in terms of efficient or means-ends causality. On this model, we think we can predict the future with exactitude (given enough information, all things being equal, etc.), and even God becomes merely a watchmaker who set off the chain of causes that is the universe.

So if technology isn't a set of gadgets or the things people do to make those gadgets, if it's a way thinking that often occludes other ways of thinking, what then? It's worth noting that this definition solves the problems with the other conceptions of technology defined above, since technology is not just something humans do--it's one of many relationships we can have with other people and things. We might be able to judge the new possibilities technology makes possible, but not technology as such.

Most importantly, Heidegger's understanding of technology as means-ends thinking points to the need for other ways of thinking--aesthetic, cultural, social, ecological, religious, political, ethical, philosophical, kinesthetic, you name it. I'm often turned off by tech news, because it reinforces the culture of buying the latest disposable gadget instead of the development of truly important solutions. Rather than fleeing from technology as some Luddites do, we need to imbue technology with the real world technologists often ignore. I like my smartphone, but it's a far cry from the future envisioned by the prognosticators of the 1950's.

Links:
-Heidegger's essay in a more-or-less readable format
-Herbert Marcuse's essay on the political ramifications of ubiquitous means-ends thinking

A Typology of Type Systems

When talking about type systems, a lot of terms get thrown around--strong, dynamic, safe, etc.--and it's often not clear what we're talking about. Jon Skeet provides a useful set of categories in his C# In Depth, a book so good that it's not just of use for C# developers. How a language handles types is a big part of what separates it from other languages, and it largely determines how it feels to code in it. It's important to remember that languages don't have to be typed, but typing creates a useful layer of abstraction that helps optimize performance, readability, and safety.

Static vs. Dynamic Typing: A language has static typing if the type of a variable must be known before run time. Compiled languages are statically-typed, whereas dynamic languages are dynamically-typed. For instance, Pascal, C++, and Scala are statically-typed while JavaScript, Ruby, Erlang, and PHP are dynamically-typed. The main advantages of static typing are: avoidance of runtime errors, memory optimization, and IDE features like auto-completion. Dynamically-typed languages let you do things like metaprogramming and duck-typing, which helps with code reuse. Explicit vs. Implicit Typing: Static languages can be either explicitly or implicitly typed. If a language is explicitly-typed, you must declare a variable with a type, such as:

var int = 1;

If a language is implicitly-typed, you can declare variables like so:

var = 1;

The compiler can figure out what the type is from the context. I'm not sure if there is any advantage to a language being explicitly-typed. Implicit typing improve readability when you have complicated types.

Type-Safe vs. Unsafe Typing: A language is type-unsafe if you can convert completely unrelated types. C and C++ allow you to do this, and there are some odd situations when it might be useful. Type-safe languages prevent you from referencing out-of-bound array locations, converting classes to dates, or adding strings and integers, though the latter could be useful for a language geared for beginners like Visual Basic. The difference between type-safety and type-unsafety is largely a matter of the conversions that a compiler or interpreter knows how to do. If it knows how to turn a string into an integer, and it knows to try this when doing operations with integers and strings, then there should be no issue with adding "5" and 3. However, there is nothing obvious about such conversions. It makes just as much sense to a compiler to add the byte values of the two variables, where you might end up with a value like 119.

Strong vs. Weak Typing: Strong and Weak typing pretty much only refer to collection types. Object arrays in C# are weakly-typed, but other array types are strongly-typed. So, while you could add all kinds of different types to an object array, you could not do this if you first assigned a string array to an object array. This is because the reference is still a string array. I don't know how often anyone would do this, but it suggests that the language of 'strong typing' should be avoided except in rare situations.

It's interesting to note that over time C# has become more and more dynamic, with C# 4 even having a dynamic data type. We now have implicit typing to support data-centric operations through LINQ. As I discussed in a previous post, dynamic typing allows you to mold your data types to those stored in databases without necessarily knowing much about what you're going to get from the database. It's interesting that the TIOBE report categorizes languages according to whether they are static or dynamic. This line is becoming increasingly blurred, just as language paradigms are becoming less rigid as OO languages like Java and C# incorporate elements from functional programming.

Incorporating dynamic typing in compiled languages is part of a trend in which languages are becoming more complex, but also more intuitive and readable. For example, why should you have to know what type a variable will take at code-time, especially when communicating with web services or databases? Though it's easy to just start using LINQ, for example, it's important to understand the problems these developments are meant to solve and the sources of their solutions in order to truly understand them.