Sandbox Software Updates

Alright, as promised, a changelog of all of the changes up to this point:

Sandbox Software Changes: February 2010

* Added UI controls to the main pages to edit/delete blog entries.
* Added UI controls to the main pages to edit/delete/report comments.
* Adapted the QSFP BBCode parser. All BBCode should now be processed properly.
* Changed the Youtube tag to parse the URL to the video and then embed the proper code. Now XHTML 1.1 compliant. Also raised the view window size to 640x400 to match their widescreen format.
* Quotes now have a nifty set of yellow quotation marks for style purposes.
* Fixed comments to report accurate usernames for registered members rather than the old manually input names from before this site had accounts.
* Akismet filter corrected to use HTTP/1.0 requests due to changes on the Akismet backend that were throwing invalid responses.
* Fixed comments so that if a user has no icon specified the default Anonymous icon will always be displayed.
* Random quotes list has been transferred to the database and can be entered via the AdminCP.
* Fatal error output has been masked with a user presentable page that does not reveal potentially sensitive information.
* HTML markup corrections to bring the code back to valid XHTML 1.1 standards.
* Comment posting and editing forms have been made immune to double posting errors.
* Added some more TLD's to the email contact form validation.
* A whole lot of template shuffling and style nitpicking.
* New settings can be manually entered into the settings table from the AdminCP.
* Number of recent comments and recent images is now configurable from the AdminCP.
* Preview added for blog posts and comments (when editing them too).
* Query errors should now properly trigger the error handler instead of being dumb and spitting the SQL query to the user screen.
* Full emoticon support, with skins able to have completely independent sets.
* Blog posting form now has prompts to include an image for the top left of the posts. Either upload a new one or pick an existing image that's already been uploaded.
* Archives section merged into the calendar box. It was getting too large and unwieldy.

There's still quite a bit left on the agenda before there can be a new release though:


Overhaul the skin on the AdminCP.
Make sure all AdminCP controls are in sync with the new front page UI updates.
PUBLIC installer needs updating something fierce now.
Option to fix comment count values that might get out of sync.

Downloads Module

Primary image support.
Multiple secondary images. (aka, like Nexus)
Better presentation of the file boxes.
No longer need hackish protected folders. Use real user account access now. > PRIVILEDGED at least to see them.
Add comment support now that there's a global mechanism that works for them.

Image Gallery

Change folder navigation to a tree along the top the way QSFP currently does. Or mimic the files module layout for categories.
Put a border around each thumbnail on a page.
Investigate the use of Lightbox type scripts to expand thumbs to larger sizes.
No longer need hackish protected folders. Use real user account access now. > PRIVILEDGED at least to see them.

Provided I don't end up bored from lack of motivation on this, it shouldn't take too long. None of it requires leaving the test posts in place any longer so those have been removed now.
RIP United States of America

July 1776 - November 2012.

« Missile Defense Logo
Lost Time »

Posted on Feb 27, 2010 4:31 pm by Samson in: | 88 comment(s) [Closed]
Sounds like it's going to be a great release when it's ready. Can't wait to see it. :)

On a seperate note, I was starting to enjoy the test posts so I'm a little sorry to see them gone, but you did warn us from the onset that they'd be deleted when you were done with them. *shrug*

Edited by Conner on Feb 27, 2010 5:38 pm
Anonymous [Anon] said:
Comment #2 Feb 28, 2010 7:50 pm
▲ ▲

Well that's nice and cryptic. Cool, but cryptic.

:lol: I'm going to have to second that, it's interesting but cryptic. Best guess would be that our cryptic anonymous friend is trying to indicate that they agree with me, but only they would really know for sure.

I am a test comment to test clickable emoticons.

:unclesam: :headbang: :facepalm: :ghostface:

Our squirrel alien overlords resent their lack of an emoticon.

Also, I have now used the preview function. Yay. The yellow text is watching you.

Apparently Anonymous believes that you need the Tri-Force.

Or maybe you're a member of the Hojo samurai clan?

That must be it, you're a test comment and I'm a Hojo samurai. :lol:

And Anonymous thinks I need the Tri-Force. Which I had to go look up in Google to know it was something from Zelda. Nope, I never played Zelda. So those of you with your chins scraping the floor in disbelief can pick them up now :)

:lol: I vaguely remember having played Zelda a few times many years ago, but I couldn't honestly say that I remember anything from that game, so I'd have to assume that I didn't find it enjoyable/interesting enough to have played enough for it to have become memorable. But then, I was way more into games like Galaga/Galaxian at back in '87, and I spent much more of my gaming time in arcades than on game machines too. :shrug:

Had some hiccup there for a bit. I'm trying to substitute in a new template engine which got bitchy about the file path. It's working now though and I bet nobody really notices the fact that half the page layout is no longer being queried from the database :)

Nope, really can't tell. Other than the rather interesting warning at the top of the page "Ongoing testing on moving the XHTML templates to a non-database driven solution. Expect the site to be sporadically available while this is going on." ...So, what's the difference, or, perhaps even better, what's the benefit of switching from database to (presumably) flat files? I had been under the impression that the big push for migrating stuff from flat files to database had a solid foundation.

Well, it's like this.

Having templates stored in the database is a pain to maintain. Fragments of stuff is all over the place, much like you see in QSFP. No surprise since Kiasyn used the QSFP template engine to drive the software :)

Generating a new skin would mean a crapload of new templates stored in the database. Assuming that were even possible with this software. It currently doesn't even support multiple skins. But if it did, then it would also require another table in the database to keep track of what skins were installed.

With the template engine I found, everything needed to format the display of each page is stored in text files. The code simply reads the data in, does it's magic foo on turning your HTML layouts into useful text, and out it comes to the screen. It's far simpler to maintain, but it does take some getting used to in order to force it to behave with the way the software is currently designed.

Kiasyn was all hot to trot about Smarty, but I didn't like how that worked. It required far too many files and had to have a world writeable directory for compiled template code. XTemplate is the one I found, and it parses on the fly. It seems quick so far, but I've only converted and removed the index page and the spam control menu from the database.

The only pain in the ass so far I see is that for each module that loads I have to allocate memory for another instance of the class. There's no way to tell it to parse certain blocks from different template files. I don't know enough about how it works to know if I could modify the class to do that. I got the thing here: so maybe if someone more savvy wants to look at it, it would be nice to be able to instantiate one instance instead of one in each module.

"Magic foo" ... is that chinese food that shows up at your door instantly? or an asian wizard? :wink:

Here's something I just noticed.

bottom of page said:

Page generated in 0.0884 seconds using 15 queries.

Is this new? I'm pretty impressed. I think the images take about 5 seconds to load though lol

Yeah, I added that bit at the bottom in an attempt to see how the software performs. It's twice as slow as QSFP.

On the main homepage, I see: Page generated in 0.1326 seconds using 17 queries.

Loading this blog post and all its comments, I see: Page generated in 0.0815 seconds using 16 queries.

Compared to the information at the bottom of 0.0402 Seconds - 16 queries

So for the same number of queries, it takes twice as long for Sandbox to process them. I need to figure out where I can cut into that and if there's any I can drop for other methods. The sidebar eats up quite a bit, and I found one spot I can eliminate a complex query from, and possibly greatly reduce another. But there's probably a bunch that are terrible at what they do.

Samson: Sounds good to me, I've been amongst those who've resisted going to database templates because my experience with them (in QSFP) has been a pain, especially when you're trying to make a change and don't know which template your changing ahead of time. I'll take a look at that link when I have a few quiet moments and see if I can make heads or tails of it for you. No promises because frankly I believe you to be the more savvy of the two of us, but maybe something will jump out at me that didn't for you. :whistle:

Hanaisse: Nah, "Magic Foo" is a codespeak synonym for "Automagically". ;)
Believe it or not, the page generation time thingy has always been there and it's only measuring the time that it takes for the query from Sandbox to return from mySQL, not the time it takes for you to get the page. My QSFP and Sandbox both return equally impressive numbers most of the time too, but it sure doesn't feel like it when you're hitting them from outside the network. :lol:

Edit Note: Apparently :D isn't matched to :lol: yet.. *hint, hint, Samson* ;)

Edited by Conner on Mar 3, 2010 3:43 pm
Actually it's not timing just the queries. The page execution timer starts at the moment index.php is loaded, and is marked as ended right above the final parsing statement. I'd move it beyond there, but then adding the information to the page would disrupt the layout. We'd only be talking about a difference measured at 0.00x seconds though.

I should put in code to time the execution of the queries themselves though because I really do need to know if trying to micro-optimize them is worth it.

BTW - The RSS feed has been totally converted over to the new system now.

Edited by Samson on Mar 3, 2010 3:56 pm
Magic foo sounds good, cuz it's all magic to me! I don't understand half this stuff, but if it isn't how fast the page is loading for ME, then pffft, lol

@Conner, that's because :D would be :biggrin: , (labeled as biggrin between colons) the one you have is the lol :lol:

Samson: That wasn't already there for Sandbox? :redface: I could've sworn it was... :facepalm:
Guess I'm just that used to see it on so many of the sites I regularly hit that I take it for granted anymore.
So, what sorts of things impact that query timing? Is it just how complex the query is?
I wonder if it'd be feasible to add a timer that shows how long the page takes to load from the visitor's perspective, even if it'd be mostly meaningless for the sake of maintenance, it'd be interesting.. though it might encourage folks to hit reload multiple times trying to get a better "score" too. ;)

Hanaisse: Okay, fine, then :D doesn't map to :biggrin: yet, the main point remains though. :P

Well if you'll notice now I've altered the line which is showing very clearly that the queries aren't the issue on time. Since the only other thing left is PHP parsing, it should be obvious now that micro-optimizing the DB queries will gain nothing.

Page generated in 0.133 seconds. 17 queries made in 0.0089 seconds.

Come on. 0.0089 seconds for ALL of the DB operations? Either the timer execution code is wrong, or MySQLi is that damn fast. I'll try one with the regular MySQL library and see if it makes any difference.

MySQL's supposed to be that fast, but who knows. *shrug*

For me, this time, it shows "Page generated in 0.0977 seconds. 15 queries made in 0.00855 seconds." so I guess it has oher factors too.

MySQL should blitz any query a forum or blog code can throw at it assuming the server hardware is below average and it has barely enough ram to run everything. I have seen benchmarking for it with 1000's of concurrent users hitting it with really complex queries and it just keeps on serving it up. Its some pretty neat software i think.

Yeah, I'm seeing that now, but that doesn't mean I can't try and reign some of these wild queries in. Suppose someday this software gets used by a site that has thousands of blog posts. You think they'll like having a calendar query that takes ages to complete because it wasn't restrained? :P

:lol: While the intent is admirable, I suspect the point is that even in a worst case scenario you're not really looking at queries that'll take as long as most folk's bandwidth for the images and such anyway. :headbang:

On the other hand, if you can clean up the queries so they're still accomplishing all that they're supposed to with less fuss, why not, eh? :)

Though, if Google has their way, in the next few years we'll all be on gigabit+ bandwidth so maybe my previous statement is a bit presumptuous... (For more details on this, see [url][/url]) I know that, for me, it'd be pretty much a dream come true if Google would bring a small piece of that fiber network to my house so I could tell HughesNet to kiss my @$$ goodbye! :D

It's probably obsessive of me, sure, but there's no reason for, say, the calendar to query the ENTIRE table of blog posts when it only needs to display a single month. So limit the query range to that month. Then no matter how many blog posts you have, it'll only ever take as long as one month takes to pull out. From what I've seen, it's at least twice as fast as it was before. In the world of microseconds, that hardly matters, but hey.

Also, why didn't anyone ever tell me the links on the calendar were hosed before? :P

I just spent the last few hours working out why they didn't seem right, and then finally figured out the array that loads the dates and went over the entire blog history was filling in random dates with posts from other months. So that's all better now too. I didn't notice because it never showed up as an issue in places like Google Webmasters.

BTW, Conner, that report Google issued came straight out of Webmasters. You should check it out sometime. It's incredibly useful for picking up on subtle problems you can't spot through normal use.

Edited by Samson on Mar 3, 2010 11:22 pm
Oh yeah, hey Samson ... your calendar is hosed.

I'd noticed this before, but there was never any topic to bring it up in :)
I think it's only wrong when I first log in and corrects itself after a page refresh.
And since this was taken this morning, is it really all better now? :P

<< prev 1, 2, 3, 4 next >>
Comments Closed
Comments for this entry have been closed.

Forgot Password?

 1 2 3 4 5 6
7 8 9 10 11 12 13
14 15 16 17 18 19 20
21 22 23 24 25 26 27