|
|
Friday, May 9th, 2008
Question about timeouts on select(): if anyone has ideas about this, please let me know in comments. Obviously select() is not a real-time operation; if you pass in a 1-second timeout, you cannot assume that you will get to run again in one second, since the operating system is allotting time to all the processes on the machine: in an extremely busy environment, it could be several seconds before you get the processor back. But I'm wondering whether the timeout is 1 second of real time, or 1 second of execution time -- in the very busy environment where your process does not get another time slice for more than a second, would select() continue to wait on the files you passed in until it had waited for a second? Or would it return immediately? (select() as it is used in this post should be read to mean "select() and poll()," since I'm assuming both API's behave the same in this regard. Who knows, maybe they don't! But that seems unlikely to me.)
posted afternoon of May 9th, 2008: 2 responses
➳ More posts about Programming
|  |
Wednesday, May 7th, 2008
So last night I was maintaining code for a program which loaded a helper program for handling data files. Before it executed the helper program it would check the sum of the binary, I think because certain instances of the helper needed special handling; if the sum was not on a list of recognized values, the program would log an error and exit. Unfortunately the helper program was not stable and was being recompiled frequently; every time this happened I needed to edit the list of recognized sums, which was hard-coded into the main program, and recompile the main program. I was embarrassed about such a stupid bit of code being in the program so I was editing, compiling, and distributing the main program without mentioning it to anybody. What a stressful dream that was! (Sort of ties everything together in a way, that I woke up humming Bessie Smith's "Gimme Pigfoot", which was in Gertrude Sturdley's post this week and which I was working on a fiddle version of last night.)
posted morning of May 7th, 2008: Respond
➳ More posts about Projects
|  |
Friday, May second, 2008
I said yesterday that I wanted to reminisce some about starting the blog. Well: I think I've written most of this before, but here goes. In 1999 or thereabouts, I decided I wanted to have a website, and that it should consist of a notes about what books I am currently reading. I cajoled my then-employers to give me some space on their server, I bought a domain name from Network Solutions, I wrote a couple of pages. The site went live the same evening Ellen's writing group held a reading of their work at Cornelia Street Café; we announced the site's launch, fun. Over the next couple of years I wrote sporadically for the site; occasionally came up with some really interesting pages. (My notes on reading Faulkner's The Hamlet are one of the most popular pages in all of READIN -- nearly every day brings a couple of Google searches for "The Hamlet by Faulkner", which I guess must not have a lot written about it on the web. It is probably my favorite Faulkner book, largely because the process of writing notes on it went so well.) The technology supporting READIN at this point was Notepad to compose posts and a Visual Basic™ program to compile them into nicely formatted HTML. In 2002 I started noticing blogs (I think the first one I read was Tom Tomorrow's This Modern World, which I found while searching for an online publisher of his comic), and getting interested. I didn't realize for a while that blogging was what I wanted to do -- I was hardly maintaining READIN at all anymore, and I didn't make the connection between my web site and this new technology. (I'm slow that way.) But after about a year of reading blogs I decided to give it a try, and in the space of about a week hacked together an ASP script to render pages. And this journal was born. And, well, this is one of the main things I do for fun nowadays. It gets more and more interesting as I learn new ways I can use the technology. Hope to keep it going for a long time.
posted evening of May second, 2008: 3 responses
➳ More posts about The site
|  |
|
 I like it.
posted evening of May second, 2008: Respond
➳ More posts about Pretty Pictures
|  |
|
This week I have been making an effort to add illustrations to my posts and to my sidebar links. I'm pretty happy with how it's looking and intend to keep doing that -- it seems to make the page a lot more visually engaging. Also thinking of trying to find a graphic that would fit nicely in the upper right-hand corner of the page, where there is a lot of empty space.* Somewhat related, I've been doing a lot of work on the interface for adding new posts and editing posts, so that I'm able to see how the post will render as I'm adding it. Would like to do something similar for adding/editing the daily "Of interest:" links, but that is going to be a bit more involved.
 *Update: Found one! How do you like it?
posted evening of May second, 2008: Respond
|  |
Thursday, May first, 2008
 I totally forgot to note the passing of another year blogging, which happened last Friday. I have been keeping this journal for 5 years, now! (At roughly 240235 posts/year.*) That is a long time.
I feel like writing some reminiscences of starting this journal, but not right now. Maybe on the weekend I will. (In the meantime, here is an early post about my motivations in starting a blog.)
 *Forgot: the id number of the latest post is not quite the same as the number of posts, since there have been a couple of deletions here and there. (It's very front-loaded though: in the first four years the average number of posts is 176, in the fifth year there are 473 posts. I wonder what the sixth year will bring?)
posted morning of May first, 2008: Respond
| |
Saturday, March 29th, 2008
Permit me to wax geeky for a moment: last night I added a new feature to the site, which involves dynamic loading of page elements. Fun! About half (a little less) of the size of this page (ie, roughly 20K bytes) is the blogroll, on the right-hand side of the page under the "Where to go from here" heading -- which you might not think to look at it, since most of the data is hidden when the page loads and only shown when you click on the category headings. This means that whenever a person or a robot downloads the page, 20K of data is sent that is probably not going to be displayed or used. I've been trying for a while to figure out how to only send it to people who are interested in it, i.e. not to robots or to one-time visitors who come here from a Google search for a book their class is reading, which together account for the great majority of page views. Last night I came up with a pretty seamless solution: I recently implemented a sticky blogroll, using cookies to ensure that once you have clicked a blogroll category header, the category will remain visible when you reload the page. So it's easy to check whether a user is a repeat visitor who has in the past looked at the blogroll -- since the blogroll is stored in a file on the server that gets read at load time, all I had to do was create a truncated version of that file (containing only data for the default visible categories) and include that instead if the relevant cookie was not set. But then when a new visitor decides s/he wants to look at the blogroll, and clicks on a category header, how is s/he going to get data? Well in the truncated blogroll file, the empty headers are linked to a Javascript function called load_full_links, which uses XMLHttpRequest to download the complete blogroll -- so the first time you click a category you will see about a 1-second delay. But then your cookie is set, so going forward the blogroll is loaded with the page. I think it's a pretty nice bit of design/programming and I'm looking forward to using a similar algorithm for other pieces of functionality.
 Potential problems: - I have tested this with Firefox and Internet Explorer. I'm not sure how it will work with other browsers, and it will definitely not work in a browser where Javascript is disabled. This doesn't seem like a big deal to me but if it causes trouble for you, let me know and I'll try and come up with a solution.
- Search engines will no longer index most of my blogroll. This is, on one hand, good -- I have seen referrals from searches that hit an item on the blogroll, which generally seem like my page is not what the searcher was looking for -- and on the other hand, possibly not ideal -- Google and Technorati both pay a lot of attention to outgoing links.
posted morning of March 29th, 2008: Respond
| |
Tuesday, March 18th, 2008
Handy to know: if you write to stderr from a CGI application (note: when I say "CGI application", I don't actually know what that means -- what I mean is, an application invoked by the http server in response to a GET or POST request and which uses environment variables to get information about the request), the output will go into the http server's log file. At least it will if the http server is Apache.
posted afternoon of March 18th, 2008: Respond
| |
Sunday, March 16th, 2008
Last night and this morning, I added another feature to the site, which you might not notice if you don't use my blogroll often; but it's a neat feature so I'm going to tell you about it. The expandable categories in the blogroll will now remember between invocations of the site, whether they are open or not; so if you click on "Blogs | Politics" and then visit Obsidian Wings, next time you come to READIN, the Blogs | Politics links will be open. A little thing but I had been wanting to do it for a few months now. Ideally I would like to have the categories be prefaced by a "+" or "-" character to indicate that there is material beneath them; right now you just have to know that something is there "because it looks like a blogroll", which doesn't seem ideal. I think this is within my Javascript programming abilities, look for it to happen sometime in the next month or two. Also, I improved the formatting of the indentations in the blogroll. I had been doing it with blocks of characters; instead I am now using <span>s with padding-left. This means that long link titles wrap with a hanging indent rather than wrapping to the leftmost column; much prettier.
posted morning of March 16th, 2008: Respond
| |
Friday, February 15th, 2008
The fix is in -- the server is using poll instead of select, a new version has been built and delivered to the client, it can handle loads of clients. Here is the long and short of how you do it (without error-checking, which is dull*): The Old Code
void select_files(int *fds, int nfds)
{
int i, maxid;
fd_set rset, wset;
timeval tval;
FD_ZERO (&rset);
FD_ZERO (&wset);
maxid = 0;
for (i = 0; i < nfds; ++i) {
FD_SET (fds[i], &rset);
FD_SET (fds[i], &wset);
if (fds[i] > maxid) maxid = fds[i];
}
tval.tv_sec = 5;
tval.tv_usec = 0;
select (maxid + 1, &rset, &wset, NULL, &tval);
for (i = 0; i < nfds; ++i) {
if (FD_ISSET(fds[i], &rset))
read_file(fds[i]);
if (FD_ISSET(fds[i], &wset))
write_file(fds[i]);
}
} The New Code
void poll_files(int *fds, int nfds)
{
int i;
pollfd *pfds = (pollfd *)
malloc (nfds * sizeof (pollfd));
for (i = 0; i < nfds; ++i) {
pfds[i].fd = fds[i];
pfds[i].events = POLLIN | POLLOUT;
pfds[i].revents = 0;
}
poll (pfds, nfds, 5000);
for (i = 0; i < nfds; ++i) {
if (pfds[i].revents & POLLIN)
read_file(fds[i]);
if (pfds[i].revents & POLLOUT)
write_file(fds[i]);
}
}
In order to take advantage of the newly accessible file descriptors above 1024, you will need to add these lines to your /etc/security/limits.conf file:
(username) soft nofile 1024
(username) hard nofile 4096
I chose 1024 for the soft limit since most apps are not interested in the high number of files, and 4096 for the hard limit because I read on some message boards that performance will degrade above that number. Feel free to choose other values. You then need to make the following calls from your code (or call ulimit from the script that starts your application):
struct rlimit nofile;
if (getrlimit (RLIMIT_NOFILE, &nofile) != 0) {
fprintf (stderr, "Could not get NOFILE");
exit (1);
}
nofile.rlim_cur = 4096;
if (setrlimit (RLIMIT_NOFILE, &nofile) != 0) {
fprintf (stderr, "Could not set NOFILE");
exit (1);
}
 *If you're interested in the error-checking code, drop me a line -- I just don't feel like typing it out right now.
posted afternoon of February 15th, 2008: Respond
| Previous posts about Programming Projects
Archives  | |
|
Drop me a line! or, sign my Guestbook.
•
Check out Ellen's writing at Patch.com.
| |
