Maneater/Django development lessons, Part 2

Through a series of posts, I’m counting down to a public test of the new Maneater Web site by the end of the weekend. We’re hoping to launch Tuesday.

In the first part, I introduced the project and talked about the history. This is part two.

This is mostly a technical writeup. For those that aren't programmers, the final post (Part 4) is a general "layman's terms" overview of new features on the site.

---

Square One

In late December, frustrated with a poor development workflow in Drupal, I mulled it over with Carolina Astrain (our online editor) and decided to go back to Django. I did this on the condition that I could fashion a functional demo in four days or go back to the Drupal project. I promised to have the site (regardless of platform) done by February. A big dare on my part, but necessary in my book: I needed hard deadlines for results. This was getting done whether I liked the workflow or not.

Over the course of hosting switches for the Django and Drupal projects, all of the old code from the prior Django project was lost. But this didn't really matter because Django -- still in development -- had gone through several major revisions and backwards-incompatible changes from the version Chase and Brian had worked on.

Again, this was fine because the site barely worked in that incarnation -- we nearly rewrote it from the ground up at one point, until the project stagnated.

So, I started the arduous task of creating a full-fledged news outlet Web site from nothing. Did I mention that I didn't know Python and that I was in way too deep the last time I worked on the old Django project? Yes, I was learning this on the fly. (I actually wish I hadn't lost Chase and Brian's code; I did not understand a bit of it back when I was responsible for it, but I believe I could learn a lot from that now.)

Django is a framework -- not so much as a "program everything" programming language as much as a programming language that does a lot of your workflow for you. Having such a versatile framework environment is probably the only reason I was able to do this with such little help.

---

And we're off

Django uses a "design pattern" (workflow) that is split up into three "layers". Most frameworks are. Django uses a very pure three-tier method: model, view, template. Or in layman's terms: what sort of information you have in your database, the logic that determines what stuff to get from your database and processes it, and the templates that take that "view" and turn it into pages, feeds, and other files. (For those programmers that know of the "Model-View-Controller" pattern, the "view" represents your controller in doing the logic, and the "template" represents your view.)

I think it's a nice system, because you can delegate your tasks based on those layers and so long as your team members know what to expect from people working on another "layer", they don't need to know the innards of the other layers.

But of course, I'm working on this more or less on my own (save for some templating help from Justin Myers) so that layer separation doesn't necessarily save me except for the fact that it's really well-suited for learning on the fly. Think about it: instead of worrying about every aspect of a site at once (overload!), you can focus on and learn specifically how to model your data and finish that, focus on how to create views for your data and then finish that, and then learn the nitty-gritty on templating.

Let me tell you one thing, though: Frameworks like this are designed to be quick-to-build. It's a framework, in literal terms, where you build off of a mostly prefabricated structure. I can imagine designing fast workflows around this, with the ability to churn out very diverse applications (don't just copy and paste the same code between projects) at the drop of a hat. I'd be happy to learn a few other frameworks and use another MVC/MTV pattern in the future.

---

Next Top Model

The beauty of a framework like Django is the data modeling features. If you've used an entity-relationship diagram (ERD), you already know more about this than is necesary. If you can list out the types of metadata for something...

  • Issue
    • Volume (Integer)
    • Date (Date)
  • Staffer
    • First Name (Text)
    • Last Name (Text)
    • Position (Text)
  • Article
    • Title (Text)
    • Date (Date)
    • Issue (Issue)
    • Section (Text)
    • C-Deck (Text)
    • Byline (Staffers)
    • Body (Text)
  • Photo
    • Date (Date)
    • Byline (Staffer)
    • Photo (Image)
    • Cutline (Text)
  • ...you more or less already have your database set up and an admin interface automatically made for you. This example is a gross simplification of the Maneater's basic data models. (If you're really interested in learning what models look like in code, see the Django Book.)

    The best part about Django's data models is fields that represent images you upload (for photos and graphics) and regular files you upload (for PDF pages and podcast audio files). The admin panel automatically takes care of uploading files to a folder you specify.

    That example admin interface uses different example data, but visually it's the same admin panel the new Maneater site uses. And I'm assuming that the Columbia Missourian and Vox Magazine admin interface probably looks the same too.

    The automatic admin interface is one of the single most important features of Django. Believe me, it takes forever to program your own way of adding, modifying, and removing stuff from a custom database.

    Taking that CS 3380 database class at the same time as this project also propelled me along quite a bit. I cannot stress enough the importance of good, connected data models, and that class is where I got it pounded into my skull. Of all the things in the Django framework, your data models really can't be edited after the fact without massive pain -- i.e. manually editing your database. It's better to have more metadata than you need than to realize later that you left out some information.

    Create a very full outline of the things you plan on putting into the site and every bit of information that goes under them. I don't think I could start a major project in any language without doing that sort of visualization now.

    ---

    Of course, data modeling and having an admin interface doesn't give you a site that's browsable.

    Next: "Views" and templating.

    Maneater/Django development lessons: Part 1 (Prologue)

    Through a series of posts, I'm counting down to a public test of the new Maneater Web site by the end of the weekend. We're hoping to launch Tuesday.

    If you're interested in understanding the history of the project and the underlying technology that powers a fully-functional, cutting-edge newspaper site, then read up. If not, wait 'til the final post Sunday afternoon or evening. To be fair, there's a chance this ends up being a treatise on why frameworks are awesome and why I like Django so much right now.

    This is mostly a technical writeup. For those that aren't programmers, the final post (Part 4) is a general "layman's terms" overview of new features on the site.

    ---

    I'm not much of a quitter. Really. At some points last semester I still held all four of my jobs -- you know, the jobs that sucked the life out of me during the disaster semester I bombed -- even though I said I'd quit half of them. That's a dead giveaway to me. (I've since made good on my word to only work two of those jobs and on a more limited basis than before.)

    But I also haven't been very good at finishing anything last year after say... Spring? I've recently bitched and moaned to a few people about how I have added absolutely nothing to my web, photo, and writing portfolios since right around November 2006.

    To start this new year I decided, "to hell with it all, I'm going to finish things right now." And I jumped headfirst into this project, aiming to finish it once and for all.

    Backstory

    From 2005 to 2006, Chase Davis and Brian Hamman worked on building news sites based on Django, a Python-based framework. The Maneater site they built was the first Django site they worked on. They later launched successful Django-powered sites for the local Columbia Missourian and Vox Magazine.

    The Maneater site never got off the ground. Although the site was complete, the project was marred by one false start after another after another due to its knack for crashing and burning. I came in and thought I'd solved the server load issues and we launched again. The site again buckled under any standard load. Over the course of the past two years, the project became a textbook example of vaporware.

    Goodbye Django, Hello Drupal

    I stepped down from my post as Online Editor this school year so I could put more focus on school and have fewer outside responsibilities. (I did mention that disaster semester, and I did not want to repeat those results.) We assembled a new Web team and after much deliberation we decided to scrap the Django project and build a site in Drupal, a PHP-based content management platform. We developed a new template and actually had a decently functional site going.

    I wasn't a big fan of Drupal, as it represented a very static way of using data. Everything was represented by "Nodes," which sounded simple enough to extend into different types of data, but became a nightmare when trying to create the type of cross-linking data models I had in mind for the Maneater. But I wasn't in charge, so the argument became moot; I just put my faith in our project lead and our team and went with it.

    We stuck with the Drupal project until our project lead flaked out and I became frustrated with the prospect of finishing programming the site myself and importing the archives from the old Maneater site -- Drupal's database structure is not the friendliest once "nodes" are extended. (For reference, the old site's database table of articles alone weighs in at over 27,000 rows and is larger than 80MB.)

    ---

    Next: Going back to Django and creating the initial features, data models, and admin site.

    Darkroom Adventures

    I've been sick since Sunday and it's sucked quite a bit. It's been (not unlike the stock market) inconsistently getting better and worse and better and worse over the past few days. (Not necessarily in that order.) But I've been downing orange juice like no other, and hopefully the combination of rest, juice, tea, and medicine has started turning the tide. Honestly: who gets sick to start off a semester of school?

    After an afternoon nap, I was feeling well enough this evening to spend some time in the craft studio darkroom. Because hey, I finished this roll of film almost three weeks ago and I wasn't about to get lazy and pay someone else to do it. And I get cabin fever all too easily; I can't think of more than five instances where I spent more than 18 consecutive hours at home.

    In my negatives binder, the last page is dated "Dec 2006 - Jan 2007." Damn. It's been a while.

    From the Flickr page:

    Gabe

    Gabe is THE most adorable kid around.

    I took this picture, and then he ran up to me and tugged at my shirt saying, "Let me see! Let me see!"

    "It's not a digital camera," I told him. "It's film."

    He walked away with a dejected "Oh..."

    I sighed. "...I guess you don't even know what that is." None of our family friends still use film.

    Kids these days.

    I love the bordered negative carrier I was using, by the way.

    Some selects from the Facebook set of New Year's photos:
    Toasting
    Group portrait

    Those shot glasses were made out of frozen Kool-Aid, by the way.

    And oh, what the hell. A photo of Jenn's cats that we're taking care of right now, because it's the first thing I've shot with my good digital camera in almost a month now.
    Kitty Love

    ---

    In other news, I checked out House of Leaves from the library today, because Glenn mentioned it to me and it seemed like something I'd enjoy. Read up on the formatting of the book and you'll see why I'm so interested in it.

    I've already got a few things to say and a few quotes to hand out. Stay tuned.

    A Resolution

    It's 21 days in, but I think I finally have a resolution.

    I need to shoot more pictures. Something about the past two weeks has been nagging at me to do so. Opportunities, messages, conversations -- everything's telling me that I miss the days of shooting like a madman.

    So the moment I finish doing all of this crazy website development for the Maneater (lord help me) and some of these other freelance jobs I have going on, I really really really want to start taking up photo assignments again. Of course that's contingent on finishing that major project first.

    I have a camera and lenses -- worth too many months of rent -- that I worked so hard to get... And I haven't even used it more than once or twice in two months. And that just sucks.

    If all fails, it'll give me more things to post on this ol' blog of mine: I'm averaging 1-3 things per month right now. I know I love the distraction of reading other people's blogs, so I'm sure (or at least, I'm hoping) it'd be appreciated.

    In the meantime, for the couple people who haven't already seen 'em: Some of the photos I got developed today. And the full album's here on Facebook, publicly accessible so you don't need an account there.

    $7 roll of film in a $5 cheapo camera = priceless.

    Lauren

    Moody Sara

    Intensity

    Intensity is the word this year because I've been bringing a lot of it. I've got an attitude like "no more bullshit, I'm getting things done." I've been working for real, instead of slacking off. I've been learning new things. I've gotten out more. I've been more brutally honest. Or maybe just more brutal. Confrontational. I seem to have lost some (or a lot?) of the restraint I had.

    It's come to my attention this week that I'm prone to ranting again. That my verbal assaults are more piercing and drawn out than they'd been. I've had no less than five conversations this week in which I probably came down too hard (with one of truth, frustration, anger, or depression) on the ears that had the misfortune of being tuned to my voice.

    Did I really just tell her that?

    Did I really just go off on him?

    Maybe it's all of this work that I'm trying to get done before school starts up again.

    It sure isn't the alcohol because of my relative sobriety for most of these conversations and the fact that this hadn't happened before in similar circumstances.

    Maybe it's just the culmination of a lot of things that have built up for weeks, months, and years for me.

    Whatever the reason, I'm evidently more high-strung than I used to be.

    ---

    A year ago I sat at my favorite coffee shop with a few friends of mine, joking around about growing up and getting old. Joking around about that "bitter old man" stereotype.

    "Maybe I’ll grow up to be bitter," I told them, "just to spite all of you."

    I said this with the biggest, most amused smile on my face.

    "Tigas, I can’t possibly picture you ever being bitter," one of them said. The tone in her voice as if I was winning the yearbook award for "Least Likely To Be A Bitter Old Man."

    I was happy and fine then, in spite of the shit I'd been through. And I really can't tell you what's changed between then and now.

    Every so often I hit these low points that seem completely unrelated to anything in my life. It's a nonsensical feeling along the lines of "not having a very good time" that pops up at random occasions when (as far as I know) most of what I've got and what I'm in the middle of is as good as I could ask for. Family, friends, work, school, it's all going pretty well right now -- and how often has that happened for me in the past what, four years?

    And that's part of the reason I feel like I need change so much: especially with relation to how my life is going, I don't think I'm enjoying myself as much as I used to. I have a lot of what I want, but I don't seem to want what I have. It's like I don't really feel like myself in these places and circumstances anymore.

    On and off for months I've become worried that (self-prophecy be damned) I'm turning jaded and bitter at the world.

    Twelve days in and I'm already wondering what this year might turn out to be.

    ---

    I know I only wrote the vague feeling and the sense of these things and not any underlying reasons. This was intentional. And there are no guesses to reasons here, either. For everyone's sake, I'd rather thoughts like that usually stay off the record.