SiteCrafting Blah Blah Blog
May. 14, 2008 at 4:15pm
Dumping Duplicates
Deleting all semi-matching rows in MySQL
One of the more recent additions to the SiteCrafting CMS arsenal is a comprehensive error logger, tracking all PHP and MySQL errors (by default... other error types can be created on a case by case basis) that occur in new sites we build. Errors are stored in our own intranet system with a timestamp, error body and a site ID (assigned to each client at a different stage of our project workflow). The table looks something like this:
+----+---------------------+---------------+---------+
| id | logTime | text | project |
+----+---------------------+---------------+---------+
| 2 | 2008-05-14 14:42:15 | A PHP Error | 1 |
| 3 | 2008-05-14 14:42:26 | A PHP Error | 1 |
| 4 | 2008-05-14 14:42:34 | A PHP Error | 1 |
| 5 | 2008-05-14 14:42:47 | A MySQL Error | 1 |
| 6 | 2008-05-14 14:42:56 | A MySQL Error | 1 |
| 7 | 2008-05-14 14:43:05 | A PHP Error | 2 |
| 8 | 2008-05-14 14:43:10 | A PHP Error | 2 |
| 9 | 2008-05-14 14:43:21 | A MySQL Error | 2 |
+----+---------------------+---------------+---------+
8 rows in set (0.00 sec)
Obviously sometimes we get duplicate errors coming through. Aside from being mere mortals who aren't always fast enough to correct an error before it recurs, one of the first stages of debugging is to try and replicate the error. These are often pretty easy to manage. It's pretty trivial to search for all matching errors, check them all, and delete them. Sometimes, though, this just doesn't cut it. Like when there are a few different errors with 10,000 occurrences apiece.
Read moreComments (0) | Posted in Coding Techniques, MySQL by Joe Izenman
Mar. 31, 2008 at 11:40am
Got API?
An API reference does a method's body good...
gotAPI.com is one of the most useful online resources I've come across, primarily because it places resources spread all over the internet into one simple site. I've been using this for quite some time, and have for the most part I have taken its usefulness for granted. Then it occurred to me that I might not be the only one that could find this tool useful (I know, it was a big 'DUH!' moment). So now I will share this gem with others...
Read more
Comments (1) | Posted in ASP.NET 2.0, CSS, From the Workbench, Javascript, MS SQL Server, MySQL, PHP, Review, Software Engineering, XHTML by Nick Williams
Nov. 14, 2007 at 3:48pm
PHP Patterns, Part II
The DAO and VO Patterns
In this installment, we will be looking at two patterns that have been 'borrrowed' from Java. If you've had any development experience with J2EE, you are probably well aware of how handy Data Access Objects and Value Objects can be. If you haven't, don't fret! This article was written especially for you!
If you've never heard these terms before, you may be wondering why I have chosen to group them together within one article. The simple explanation is ... well you'll see. For now just accept that they go hand-in-hand, much like salt and pepper or peanut butter and jelly or <insert clever combination here>.
Excited? Let's dig deeper...
Read moreComments (4) | Posted in Coding Techniques, From the Workbench, MySQL, PHP, Software Engineering by Nick Williams
Oct. 26, 2007 at 2:05pm
Apache, MySQL, and PHP on Leopard (Mac OS 10.5)
I just finished installing Leopard on my computer, and my first impressions are that it's very slick and well thought out. One of my necessities as a developer is that I must have a webserver running on my personal computer, so I was dismayed when it wasn't functional after the upgrade. The main reason is that Leopard uses a different version of Apache than 10.4 did, and so some things get wonky. But it's easy to fix. Apache and PHP are included in Leopard, so the only thing missing is MySQL. To install that, go to MySQL's site, and download the latest copy for OS X. It's incredibly simple to install.
After that comes setting up Apache. Open up a terminal window, and type in "sudo pico /etc/apache2/httpd.conf". (Note that you must be a computer administrator to access the files in etc/) OS X 10.4 had Apache in /etc/httpd/, and that's part of why it didn't work after the upgrade. Find the line in httpd.conf that looks like "#LoadModule php5_module libexec/apache2/libphp5.so", and remove the # sign at the beginning. Then search for AddType, and put the following somewhere around it.
AddType application/x-httpd-php .php
AddType application/x-httpd-php-source .phps
Save the file, and exit. Then open up System Preferences in Finder, and click on Sharing. Then turn on Web Sharing. That's all you need do to to setup a webserver on Leopard, or upgrade from OS 10.4. You should be able to open up a browser, and enter http://localhost/YOUR_USERNAME/ and see that the server is running.
Comments (30) | Posted in MySQL, PHP by Dave Poole
Oct. 3, 2007 at 3:59pm
A GUI Mess, or a Productivity Booster?
What's your take on the database admin debate?
While writing up a review on a database tool I discovered today, I was inspired to spark a discussion about database GUIs in general. The value of GUI tools for administering database systems like MySQL has been a topic of much debate.
Read more
Comments (6) | Posted in Deep Thoughts, MS SQL Server, MySQL by Nick Williams
Sep. 6, 2007 at 4:45pm
Adventures in Database Migration Pt. I
Fie on commas! Fie!
SiteCrafting is in the process of phasing out some of our older servers, and as an added bonus, the clients hosted on those servers are getting a MySQL jumpstart, leapfrogging over 4.1 to go straight from 4.0.24 to 5.0.32. Tragically, it's not quite as simple as dump | import. This is what I get for bothering my bosses for a few weeks not long after coming aboard about how nice stored procedures, updatable views, and triggers could be.
"The wonderful thing about standards," a wiser person than me once said, "is that there are so many of them." That's not the whole of it, though. One good thing about standards is that there are certain features one can generally rely upon to work, translate, port, etc. Assuming one works within them, rather than taking advantage of loopholes allowed by their not-entirely-compliant-but-we're-getting-there-and-anyway-isn't-this-way- easier-and-faster software. When people don't (and I'm not entirely innocent here), you run the risk of turning your simple upgrade into a serious project when your favorite software decides it's time to comply a bit more.
Read moreComments (5) | Posted in MySQL, Web Hosting by Joe Izenman
Feb. 20, 2007 at 2:53pm
Optimizing GROUP BY with Multiple LEFT JOINS
Creative use of temporary tables
At SiteCrafting, I enjoy working with a large number of different projects, each with their own requirements, technology, and problems to be solved, unfortunately, I sometimes forget about past solutions, until after I have finished writing a piece of code. Such is the case with a query that was eating up some serious processing time.
The problem was with a GROUP BY query with LEFT JOINs to several other tables and summing up totals from those joined tables. This query was taking about 4 minutes 45 seconds to run, and worse yet, it was affecting searches which had nothing to do with that query, and probably also eating up precious memory and cpu resources.
Read moreComments (0) | Posted in From the Workbench, MySQL, PHP by Ken Foubert
Feb. 2, 2007 at 10:53am
Query conservationist
The fewer the queries the better
I always enjoy trying to do my work creating web applications using the fewest number of database queries possible. It's kind of a pride thing, I guess. That and I suppose it makes to improve an app's performance and reduce the load on a server. If you're into that kind of thing. ;)
A technique I use subtitutes the temptation of using queries whose results call queries and instead uses only two.
Read moreComments (3) | Posted in Coding Techniques, MySQL, PHP by Kevin Freitas
Oct. 27, 2006 at 11:34am
Cross Table Content Search
One of the more interesting adventures working with data storage is trying to aggregate information meaningfully from multiple very different data structures. Imagine you've got a website filled with content (say, a few hundred pages). All the content is stored and output dynamically - who wants to create and maintain 500 static html pages, anyway? And of course, you need a bit of variety, so all this content is spread across five different page designs, each requiring its own template and data structure.
Now you say you want to search your site? All the content? And you want the results all together in one big happy sorted-by-relevancy list? How on earth am I going to do that?
Well... like this.
Read more
Comments (0) | Posted in MySQL, PHP by Joe Izenman
Oct. 24, 2006 at 9:41am
Group By Queries for MS SQL Server
Applying MySQL Wizardry to MS SQL Server
Back in the day, we worked on developing a specialty application that was basically a lead generation system with a database that stored over 200,000 records, with a potential for a lot more. The web application displayed numerous reports that calculated totals from disparate sources. We discovered that once our client began adding all their data that those reports were running slower and slower and slower.
The problem was that we had one primary query to pull the records out, then, as the code looped through each record, several other queries were needed to calculate the disparate totals. That resulted in numerous database calls that slowed the entire web application. That's when Mike discovered MySQL Wizardry, that used the SUM(IF()) and the GROUP BY clause, problem solved.
Read more
Comments (4) | Posted in Coding Techniques, MS SQL Server, MySQL by Ken Foubert
Oct. 13, 2006 at 10:59am
A Piece of Query Cake
Making Inserts Sucks
If you've ever created SQL queries with PHP, you probably know what a pain it can be to create insert and update statements. I really, really (really) don't like it. As I was working on my personal site, and exploring possible frameworks to use, I came across CodeIgniter. They have a great database interaction library, especially the function for creating the insert queries.
Today, armed with only the descriptions of CodeIgniter's query helper functions, I spent 20 minutes trying to duplicate some the effect of the insert and update functions. I've never seen the code, or even used it, but I didn't have to see the code to write a similar function. Both functions take a table name and an associative array of column names and values. The update function also requires a WHERE statement, and it can't be blank. This is different from CodeIgniter, and that's so you don't accidentally reset all of the passwords in the mysql users table, or any table for that matter. And then, *poof*, the function gives you a nice sql statement.
I'll never have to write another "INSERT blah blah blah" again. Yay!
Read more
Comments (2) | Posted in Coding Techniques, MySQL, PHP by Dave Poole