LoCo Day 16

Today I got starting and ending games working. Each member of a room can signal when they’re ready to begin playing. Once everyone in a room is ready (whether it be because the last non-ready member became ready, or the last non-ready member left), and the room isn’t empty (in which case the first condition would be vacuously true), a new game starts. When it’s over, everyone becomes not-ready and the cycle begins anew.

Granted, the “game” that’s currently implemented amount to “wait five seconds for the game to end”. Woo. Sure, it’s more like a hole where an actual game can be put, which is sort of the point.

Since the game I have in mind is real-time and not turn-based, starting a new game requires starting a new thread, which will be responsible for periodically advancing the game state and sending messages to the room if anything interesting happens. Right now that thread just sleeps for a few seconds before ending the game, but it’s there. During a real game, players would affect the game state by asynchronously POSTing their moves, which the game thread would then be able to read at the next time step.

For a program being written in Haskell, I sure do have an awful lot of code living in the IO monad. I don’t think there’s really a way around this, though, since the multicast channels underlying my long polling implementation are based on TChans (so that requests have something to block on while waiting for messages), and the list of rooms is carried around inside an MVar (so multiple server threads can access it simultaneously). Most of the code I’ve been writing lately is directly tied to sending messages and/or manipulating a room, which pretty much forces IO. The game itself will have a bit of IO when it comes to queueing players’ moves, but it ought to be possible to make most of the game logic pure.

Comments Off

LoCo Day 15

Another day where I didn’t set aside enough time to get much accomplished. I addressed the problem of a user inadvertently getting unsubscribed from a room’s messages upon reloading the page by always (re-)subscribing them whenever they try to fetch messages. At worst, they’ll miss out on messages sent while the reload was taking place, but that’s reasonable.

I then started adding hooks for actually running a game out of a room. I got mostly through the code for letting members of a room signal when they’re ready to play. Right now, since there’s no actual game implemented, there’ll be a game-started message immediately followed by a game-over message once everyone is ready.

At some point I should write test cases for all the code that implements room-related stuff, but since it’s almost entirely within the IO monad, that makes coming up with good QuickCheck properties a nontrivial exercise. Doable, certainly — I did it for the multicast channel code, since there’d be all kinds of problems if that didn’t work correctly — but not nearly as easy as coming up with properties for pure code.

Comments Off

LoCo Day 14

Essentially no progress today, other than a little refactoring of the code I was working on yesterday. Clearly I failed to set aside enough time to work on things today.

Comments Off

LoCo Day 13

The overhaul of the room code is done, at least for now. Instead of there being only one room on the server, now any number of them can exist. Woo.

I think the subscribe/unsubscribe logic is going to need a change at some point, since even with the light amount of testing so far, I’ve uncovered some design problems:

  • If you reload the room page, there’s a race condition between the unsubscribe request sent by the unload handler, and the request for the actual page. Whether or not the browser gets re-subscribed for messages or not depends on the order the server processes the messages. Worse, since a non-subscribed session trying to fetch a room’s message always returns an empty list immediately (since there’s nothing to wait on), the browser starts hammering the server with long-poll requests. Subscribing needs to be made more robust; perhaps the act of trying to fetch a room’s messages should count as a subscription request too.
  • On Chromium, the page’s unload handler either doesn’t fire or doesn’t manage to send the unsubscribe request in time. Either way, the unsubscribe never happens. Presumably there’s a flaw in my JavaScript code that does this, but even then I need to automatically unsubscribe sessions from rooms if there isn’t any activity after a little while, which will be needed anyway in case a client’s browser drops off the net one way or another.

I’ll need to fix all that eventually, but for now I think I ought to work on the actual game part of this game I’m supposed to be writing.

Comments Off

LoCo Day 12

I restored the functionality I had before reworking the room code. The problem I mentioned yesterday was because I mistyped an identifier in the JavaScript code, and apparently using an undeclared and undefined identifier isn’t a problem as far as Firefox and Firebug are concerned. Contrast that with GHC, which not only flags that sort of thing as an error, but even offers suggestions as to which identifier you might have meant, sort of like suggestions in a word processor’s spell checker. Actually, it’s kind of surprising that GHC is the only compiler I’ve seen with that suggestion feature for undeclared identifiers. It seems rather obvious in hindsight.

I also fixed displaying of timestamps; now instead of showing an ugly complete UTC timestamp, the chat window just shows the time, converted to the browser’s time zone. Now that the polling fetches JSON objects instead of HTML fragments, that sort of thing is possible to do.

Now I’m working on adding support for having more than one room on the server. I expect this to occupy most of tomorrow’s effort too.

Comments Off

LoCo Day 11

I’ve almost got things working again in the current refactoring. The current sticking point is how the server now sends JSON-encoded messages to the browser instead of HTML fragments. This will be needed for implementing a game since there will be things sent during a game other than bits of webpage to display. However, my JavaScript-fu is pretty weak, so it’ll take a little more effort to figure out why things aren’t working on the browser’s side of things.

Comments Off

LoCo Day 10

Today’s roadblock came courtesy of the aeson package, or one of its recursive dependencies. I wanted to try using it instead of json to generate JSON, since the programmatic interface seemed a bit simpler and more elegant. Alas, one of aeson’s dependencies doesn’t work due to a bug in GHC. Also, possibly as a result of that same bug, the act of installing the package screwed up the rest of GHC’s package repository to the point where it could no longer find all the other packages that had been installed, and reinstalling them mysteriously broke other unrelated packages too. I finally had to nuke the whole thing and install all the packages I’m interested in from scratch.

Then, of course, I tried installing aeson again to see if maybe the repository corruption was a weird fluke. It wasn’t.

So, I sunk a lot of today’s development time messing with that. Needless to say, I’ve kicked aeson aside and went back to using json.

A third of the way into November, I must say I’m much farther behind than I expected I would be. I still haven’t worked on the actual game part of this game; it’s all been setup and infrastructure stuff. Sigh.

LoCo Day 9

Only incremental progress to report today. Shrinking the long-polling timeout from 60 seconds to 30 seconds seemed to solve the problem of Happstack killing handlers prematurely, and some of the Internals stuff in the Happstack documentation hints at being able to have some control over whether Happstack thinks a long-running handler needs to be killed or not. Worth looking into in the long run, but shortening my timeout is good enough for now.

Aside from that, most of today’s work was in formalizing the very hackish chat room code, converting it into a more general “room” concept that will better support layering a game on top of. The new code will also support having more than one room on the server. I hope to have that working tomorrow; the types are (mostly) implemented, but I still need to generate HTML from it and hook it into the handler logic.

Comments Off

LoCo Day 8

Long polling is working! I now have a very basic real-time web-based chat room implemented. JavaScript on the browser regularly polls for updates on the server, and adds them to the chat window if it finds any. The server thread handling the request blocks for a short while if no messages are available. It’s simple, but the messaging subsystem will be the foundation of client-server communications when the actual game is implemented.

The only problem with my implementation (other than the design limitations I mentioned in the previous post) is that sometimes the server returns a 500 error to the client’s long-poll request instead of a 200 or 204 based on whether any messages are available. The error seems to be getting spit out by Happstack itself, since the specific error message (“server error: thread killed”) isn’t anything I’ve written. My best guess at the moment is that Happstack imposes its own timeout on request handlers, and when it sees it hasn’t finished after a while, assumes that the handler has gotten hung and kills it. If this is the case, the solution would be to either shorting the long-poll timeout in my code, or see if there’s a way to adjust Happstack’s timeout.

I think that after fixing that issue, I’ll migrate the chat room code into a more general “room” abstraction that’ll be amenable to building the game on top of. (The chat room code is a bit hackish at the moment.) I’ll then probably make it so you can have more than just one room in existence, which could make the main server logic a bit neater too, and give some more realistic URLs too.

Comments Off

LoCo Day 7

I think I have long-polling for chat messages mostly implemented. The one missing piece is retrieving messages on the server side. The client side can send and (try to) receive messages, and the server accepts new messages.

One obvious flaw in the current implementation is that the chat room queues messages for clients based on session ID, so if someone opened the page in two separate tabs in their browser, messages would only appear in one tab or the other, since the server wouldn’t be able to distinguish the two — each tab’s requests would be carrying the same session ID. When my implementation gets more sophisticated, I’ll either need to add another ID for a particular instance of having the page loaded, or just block attempts to have the same chat room (or game or whatever) opened multiple times by the same session.

Comments Off

LoCo Day 6

Some actual progress, now that I’ve abandoned trying to get Heist to do what I want it to and switched to Blaze for HTML generation.

Then I implemented site-wide CSRF prevention. Whenver the server receives a POST request, it checks that the csrfToken parameter matches the one stored in the session for that request. If not, it stops processing the request and spits out an error. Unfortunately, if the browser has cookies disabled, that’ll also trigger the error message, since the server won’t be able to associate the request with an existing session. Either that’ll need to be fixed somehow, or I’ll at least need to detail that cause in the error message. After all, if there really is a CSRF attack taking place, no one will actually be reading the page that gets returned anyway.

After that, I implemented a multicast messaging system that will serve as the innermost component for letting browsers poll efficiently for game-related events. Internally it’s built around transactional channels, with my code dealing with creating and destroying TChans as players enter or leave the game, and with retrieving all queued messages without blocking. It’s that last bit that requires using TChan instead of the slightly simpler Chan; the latter doesn’t have a reliable way to check whether the channel still has anything left in it.

Finally, I started working on a basic chat room mechanism, to operationally test the messaging system and to serve as a starting point for the JavaScript code that’ll run in the user’s browser. I haven’t had a chance to get very far with this piece yet, but it ought to be fairly straightforward.

It feels good to be making actual progress once again.

Comments Off

LoCo Day 5

Yet another day with little-to-no direct progress, but I think I did finally get to the heart of the problem I’ve been wrestling with. I’ve asked on Stack Overflow if there’s a way around it, in case I’m still missing something, but I don’t think I am.

The basic problem I ran into is this: In Heist, page templates can only run Haskell code by invoking splices, which are defined in the context of the program’s application stack. As a result, splices can’t see anything that doesn’t directly originate in that monad. However, when using web-routes for type-safe URLs, parameters passed via the URL path itself, instead of the query string, are inaccessible from the application monad; instead, they are accessed by pattern-matching against the type-safe URL. Likewise, the session object created before invoking the web-routes part of the code lives outside of the application monad. As a result, splices can’t get at URL path parameters or the session object, and there doesn’t seem to be a way around this inherent design limitation.

The problem doesn’t seem to arise when using Snap instead of Happstack, since Snap’s routing mechanism extracts URL path parameters, gives them names, and treats them as though they were other query string parameters, in which case they can be accessed via a Snap-based monad. I imagine something similar happens when working with sessions. Since Heist arose out of the Snap project, it’s not surprising its splices are limited to getting things out of the application monad, since in Snap that’s less of a limitation than it is in Happstack. No wonder that all the examples of Heist I’ve come across hand-wave getting request parameters by assuming there’s something in the application monad to do that.

I think that means I should give up on Heist (barring an answer to the aforementioned question that points out a way around this issue entirely) and use Blaze to generate HTML instead, at least for now. Although I’m stuck with Blaze’s quasi-monadic syntax instead of something that looks like actual HTML, at least I get some extra type-safety guarantees by having the HTML templates be Haskell code instead of standalone files loaded at runtime.

I did effectively lose a couple days wrestling with this, but at least I’m coming out of it with a better understanding of what these libraries can and can’t do.

LoCo Day 4

Another horrifically unproductive day, mostly spent searching for and reading documentation and tutorials for the libraries I’m trying to use, in the hopes of figuring out the right way to use them together.

I think the mistake I’ve been making is the assumption that I want to be using a MonadReader to expose session and request information to the Heist templates I’ll be using to piece together web pages. All of the Heist tutorials and blog posts I’ve seen hand-wave the issue of accessing request-specific data in templates and splices by supposing an application monad exists that provides an operation to get at that information. I had originally taken that to mean that I should use a ReaderT to package up that sort of information and add it to the monad stack I’ve pieced together out of Happstack, web-routes, and Heist. However, sticking a ReaderT into the stack throws the compiler into a fit, since it winds up hiding some of the monad typeclasses below it in the stack. For instance, Happstack doesn’t provide the plumbing for ReaderT to lift ServerMonad operations up, and I don’t know whether that’s a deliberate design decision or an oversight. There doesn’t seem to be a feasible place to stick the ReaderT into the stack that doesn’t cause problems somewhere.

So, what I think I’ll try to do is give up on ReaderT entirely for this, and instead pass the Session object and any other resource handles I need to worry about as function arguments (possibly encapsulated in some Context object so I have only one thing to pass around instead of a bunch) until they get to the actual request routing function. In there, I’ll create splices out of the session information for the templates to use, passing them to Heist at the same time I tell it which template I want to render. This is slightly complicated by the fact that the happstack-heist module, which provides the plumbing between Happstack and Heist, doesn’t provide a wrapper for the Heist function that lets you invoke a template with extra splices. Fortunately, it doesn’t look like it’d be difficult to reimplement that bit of plumbing myself. I’m still not certain this is the “correct” way to do things, but there’s a chance it will work, which is better than my approach so far.

I had also considered giving up on Heist and using some other mechanism for generating web pages. Blaze would have the advantage of better type safety since the HTML-generating code would be part of the program, but then I’d be stuck using Blaze functions to construct the pages instead of writing something that actually looks like HTML. HSP would in theory solve this by letting you write XML directly in the program and use preprocessing magic to translate it into Haskell code at compile time. However, HSP has the crippling flaw of being effectively undocumented, other than a bit of API documentation and pointers to a ten-year-old thesis written about it. Other templating engines seem to be merely string-based, which makes me concerned about the possibility of accidental cross-site scripting attacks.

It’d be great if there were some well-documented, simple-but-not-trivial example out there of something using Happstack, Heist, and web-routes together, but I haven’t managed to come across it yet. I guess I’ll keep stumbling along tomorrow.

LoCo Day 3

Most of today was spent wrestling with figuring out how to use Happstack, Heist, and web-routes to play nicely together. All of the tutorials I could find focus only on Happstack and Heist, or Happstack and web-routes, but it’s not immediately obvious (to me, at least) how they should be put together in the monad stack, especially when you also want to throw in a ReaderT to carry session information and a couple handles to shared resources. Hopefully if I go slowly, one step at a time, I’ll be able to figure out the “correct” solution which will no doubt be obvious once I see it.

I just wish I had more to show for the hours I’ve already sunk into this….

Comments Off

LoCo Day 2

Progress is still slower than I’d like, but it’s there. Basic session tracking is working. Now when the server gets a request, it looks for a session cookie and either loads the session from the database or starts a new session. At the moment the sessions don’t do anything more than exist, but the rest of the functionality will need them to associate different requests by the same user.

It took longer than I was expecting to get to this point. I think part of it’s because I’m having to frequently consult the documentation for the different libraries I’m using, and part of it is that it’s early enough along that I’m still having to set up basic scaffolding along the way. Hopefully that means as time goes on my rate of progress will increase.

The next order of business is to start serving a real page, implement CSRF protection, and implement multicast message queues. Before getting to the actual game, I’ll construct a basic chat room sort of thing, which will exercise the messaging system. Besides, once there’s an actual game there, it can still be used to let players talk with one another.

Comments Off