htmlentities() in PHP is Your Friend
Friday, October 14. 2011
After constantly badgering a certain library calendar vendor over 2 years to fix his software's RSS feed charset issues. Personally I don't think getting raw text describing times in the form of "6:30–8:30 p.m." is all that valuable...that's just a single example. The calendar website declared no charset information. I have no idea what charset the database is in, and the RSS feed was declared as ISO-8859-1. Our website, database, and everything else was declared as UTF-8, not that it really mattered though since the raw incoming text from the RSS feed was all garbled to begin with.
Every once and awhile I'd randomly try to find an answer to the problem. I've been through using all sorts of different algorithms to solve the problem. None of them seemed to work, until one day I saw someone mention on StackOverflow (unfortunately I've lost the link) that he tried using htmlentities() to solve his problem and it worked. I thought, "It couldn't be that simple..." However, I had nothing to lose and tested it. It worked. (What???) I still don't know why or how htmlentities managed to run a translation table on the garbled input to output the appropriate values, but I'm happy! Even my attempts at REGEX were unsuccessful, though I probably was just unable to find ALL the right bit-level character code sequences needed. Apparently the translation table that htmlentities uses is pretty darn thorough! Thanks, PHP team!
Okay, so that was the first use of htmlentities(). The second one?
I realized I overlooked a severe security hole in my forms. When users did not provide correct details in their forms, I was simply reinserting the values they provided back in to the HTML form's VALUE tag (or in the case of a textarea, just rendering the value between the tags). For some reason this didn't strike me as being severely stupid at the time. I don't know why. I guess the "never reprint what your users submit to you" only made me think of "back to the DOM" - but only outside form elements. Who knows why. This let someone who actually put some (minimal) thought into it to run whatever PHP code they wanted simply by submitting a form without all required data. Escape the form element by using a standard HTML closing tag, then start writing the PHP. If you wanted valid HTML, just make sure to also include a dummy HTML input or textarea field once done. Simple. (Note: I am also in the process of re-examining CHMOD values of files and folders.)
When I went back to "fix" my stupidity, I also initially thought of using PHP's filter functions. Although they worked, they also would sometimes (depending on user input) remove certain characters. Like a bolt of lightning (while I was eating lunch) it came to me. I just used htmlentities(), why not just use it again? ...so I did. Now my forms are a bit more protected and our RSS feed is no longer displaying obnoxious characters to visitors due to the encoding mishaps of an external developer.
Sometimes PHP's little gems are so awesome...
Every once and awhile I'd randomly try to find an answer to the problem. I've been through using all sorts of different algorithms to solve the problem. None of them seemed to work, until one day I saw someone mention on StackOverflow (unfortunately I've lost the link) that he tried using htmlentities() to solve his problem and it worked. I thought, "It couldn't be that simple..." However, I had nothing to lose and tested it. It worked. (What???) I still don't know why or how htmlentities managed to run a translation table on the garbled input to output the appropriate values, but I'm happy! Even my attempts at REGEX were unsuccessful, though I probably was just unable to find ALL the right bit-level character code sequences needed. Apparently the translation table that htmlentities uses is pretty darn thorough! Thanks, PHP team!
Okay, so that was the first use of htmlentities(). The second one?
I realized I overlooked a severe security hole in my forms. When users did not provide correct details in their forms, I was simply reinserting the values they provided back in to the HTML form's VALUE tag (or in the case of a textarea, just rendering the value between the tags). For some reason this didn't strike me as being severely stupid at the time. I don't know why. I guess the "never reprint what your users submit to you" only made me think of "back to the DOM" - but only outside form elements. Who knows why. This let someone who actually put some (minimal) thought into it to run whatever PHP code they wanted simply by submitting a form without all required data. Escape the form element by using a standard HTML closing tag, then start writing the PHP. If you wanted valid HTML, just make sure to also include a dummy HTML input or textarea field once done. Simple. (Note: I am also in the process of re-examining CHMOD values of files and folders.)
When I went back to "fix" my stupidity, I also initially thought of using PHP's filter functions. Although they worked, they also would sometimes (depending on user input) remove certain characters. Like a bolt of lightning (while I was eating lunch) it came to me. I just used htmlentities(), why not just use it again? ...so I did. Now my forms are a bit more protected and our RSS feed is no longer displaying obnoxious characters to visitors due to the encoding mishaps of an external developer.
Sometimes PHP's little gems are so awesome...
jQuery and Google Analytics: Take 2
Thursday, December 9. 2010
Awhile back I had reported on my findings of how to use jQuery to automatically tag links on my site (that link to a site that is not my own) for event tracking with Google Analytics.
Although there are ways to use the pageView method and parse the data within Analytics so that it does not skew your view of overall visits/hits, I simply find it easier to mark these as events since they aren't actually page views, but events.
Unfortunately, it's helpful to see which external links are pertinent to people, and which page they're leaving your site from (and where they're going), so rather than identifying the event type itself (click), I used that field for a value instead so it could be categorized by any of these things from within Google Analytics.
The problem I was having personally was that the values I was using were not strings (in my own tests at least; the code on the blog should have worked, theoretically). I was using document.location (type: object) and passing it to the trackEvent() method, which checks for value types (and denies incorrect calls). Here is the updated (using the new Async method) code which is tested, and works:
Remember to change your own account ID in the code above!
There is no need to define your domain's actual information anywhere unless you want to track both subdomains as well as top-level domains under a single account. For that, take a look at the Google Analytics documentation. I haven't done this myself so would not be able to suggest a best-method approach for it. This code tracks any HTML-based links (links inserted dynamically with JS will not be tracked, you can do this with $(this).live('click', function()...) instead of $(this).click()) that point to anywhere NOT within your domain. If you use a redirect script as part of your domain, (ex: ...tracker.php?go=http://someothersite.com), then this will not track that page click as-is.
This code specifically defines the event as "Outbound", the category as the currently viewed page's URL, and the action field as the URL that the user will be taken to. Modify 'til your heart's content. :)
For even more accurate results, see how to delay the loading of the external site so that the tracking code has more time to accomplish its task (race conditions are fun): Google's Solution
Although there are ways to use the pageView method and parse the data within Analytics so that it does not skew your view of overall visits/hits, I simply find it easier to mark these as events since they aren't actually page views, but events.
Unfortunately, it's helpful to see which external links are pertinent to people, and which page they're leaving your site from (and where they're going), so rather than identifying the event type itself (click), I used that field for a value instead so it could be categorized by any of these things from within Google Analytics.
The problem I was having personally was that the values I was using were not strings (in my own tests at least; the code on the blog should have worked, theoretically). I was using document.location (type: object) and passing it to the trackEvent() method, which checks for value types (and denies incorrect calls). Here is the updated (using the new Async method) code which is tested, and works:
Remember to change your own account ID in the code above!
There is no need to define your domain's actual information anywhere unless you want to track both subdomains as well as top-level domains under a single account. For that, take a look at the Google Analytics documentation. I haven't done this myself so would not be able to suggest a best-method approach for it. This code tracks any HTML-based links (links inserted dynamically with JS will not be tracked, you can do this with $(this).live('click', function()...) instead of $(this).click()) that point to anywhere NOT within your domain. If you use a redirect script as part of your domain, (ex: ...tracker.php?go=http://someothersite.com), then this will not track that page click as-is.
This code specifically defines the event as "Outbound", the category as the currently viewed page's URL, and the action field as the URL that the user will be taken to. Modify 'til your heart's content. :)
For even more accurate results, see how to delay the loading of the external site so that the tracking code has more time to accomplish its task (race conditions are fun): Google's Solution
Finding all application Controllers in CakePHP (v1.3.x)
Tuesday, September 28. 2010
At some point in time you may find the need to get a list of all of the controllers within your application. It's actually quite simple so long as you don't need controllers from any plugins.
Place this code in any of your controllers and view it from the web (for instance, from the Users Controller): http://www.example.com/users/listControllers
You'll see something similar to the following:
The App::objects() method returns an array of objects of the given type, such as: 'model', 'controller', 'helper', or 'plugin' - it also accepts other parameters, such as "path" in case you do eventually need to check controllers for your plugins.
The array_diff() is there as a much simpler method to remove the AppController and PagesController from the returned results as they would most likely exist in your application regardless and aren't normally something you'd need to worry about with ACL as there are other means within Cake to handle access to them. You'll notice that array_diff doesn't return an array starting at index of 0 since it removes keys from the original array too. That shouldn't be a problem, but if it is you can always use a loop instead of array_diff() and just unset or splice the matching values.
I used the CakePHP convenience method of pr() (print_r surrounded by PRE tags), and a die() simply to show the returned results and print them to the screen. You'd probably prefer to double underscore the function name (__listControllers()) to make it a private method to the class (rather than publicly viewable via the web), and change pr() to return.
What's the purpose of this?
- Maybe you'd want to create a web interface for ACL and need to know which controllers to give/deny access to/from
- Maybe you want to create a navigation menu based on your controllers
...maybe you can think of something that I can't. :)
Alter to your own tastes. This is only a starting point.
Other useful links (check version compatibility in these resources):
Quick dessert: List all controllers of a CakePHP application
Automatically load all controllers and actions into ACO tables for ACL with a CakePHP Task
How to list all controllers
Place this code in any of your controllers and view it from the web (for instance, from the Users Controller): http://www.example.com/users/listControllers
You'll see something similar to the following:
The App::objects() method returns an array of objects of the given type, such as: 'model', 'controller', 'helper', or 'plugin' - it also accepts other parameters, such as "path" in case you do eventually need to check controllers for your plugins.
The array_diff() is there as a much simpler method to remove the AppController and PagesController from the returned results as they would most likely exist in your application regardless and aren't normally something you'd need to worry about with ACL as there are other means within Cake to handle access to them. You'll notice that array_diff doesn't return an array starting at index of 0 since it removes keys from the original array too. That shouldn't be a problem, but if it is you can always use a loop instead of array_diff() and just unset or splice the matching values.
I used the CakePHP convenience method of pr() (print_r surrounded by PRE tags), and a die() simply to show the returned results and print them to the screen. You'd probably prefer to double underscore the function name (__listControllers()) to make it a private method to the class (rather than publicly viewable via the web), and change pr() to return.
What's the purpose of this?
- Maybe you'd want to create a web interface for ACL and need to know which controllers to give/deny access to/from
- Maybe you want to create a navigation menu based on your controllers
...maybe you can think of something that I can't. :)
Alter to your own tastes. This is only a starting point.
Other useful links (check version compatibility in these resources):
Quick dessert: List all controllers of a CakePHP application
Automatically load all controllers and actions into ACO tables for ACL with a CakePHP Task
How to list all controllers
jQuery: Edit Content Inline on Dynamically Created Elements
Thursday, July 22. 2010
Unless you've been living under a rock while learning jQuery, everyone should have learned of the .live() call within the jQuery JavaScript toolkit. If you haven't, shove that rock aside - the .live() method takes the place of bind() (which is the more powerful, and base method of .click(), .blur(), etc...) for elements that are created dynamically at runtime. If you dynamically create elements after the DOM has loaded, and assume generalized JavaScript functions you've written to target these new elements (we'll presume you're targeting a "findMe" class) will work. Well it won't, unless you use .live().
Now that we have that out of the way, another one of jQuery's really great features that mimics why Wordpress is so great - all of the user contributed plugins, whether they're efficient or not, the fact that someone else has already created a solution for you, and is willing to share it with you (most likely for free) is awesome. One such type of plugin that I was looking for today, and knew I'd want to eventually use at some point, was an edit-in-place, or inline edit plugin - something that allowed me to edit content from within the DOM, such as in a DIV container (not in a textbox - or at least not initially rendered in one) dynamically.
In my case today, I was experimenting with dynamically creating elements that might need to be edited after being created. My first gut-instinct thought? Yup, you guessed it: .live(). I replaced the call to .bind() in the plugin's source and voilà! Nothing. It didn't work. The jQuery code in the plugin had been optimized beyond a point where I was comfortable to modify it further and understand what I was doing in order to make it work (which was kind of pathetic, but I digress).
So, how could I solve my little problem? Quite easily - I wasn't really thinking.
The call to instantiate an event on targeted objects after the DOM is ready is something like this:
I was scratching my head, trying to figure out if I'd have to uninstantiate all elements and reinstantiate the event bubbling throughout the DOM each time I created a dynamically created element. I was over-thinking it. Instead, all I had to do was create the event on that newly created element on the element's creation itself. Ooh, hard. I know, I know - I'm brilliant!
jQuery v1.4+ example code:
Easy and simple.
You can see how to further use and extend Arash Karimzadeh's jQuery Editable plugin, or if you'd prefer a more popular plugin for inline editing with jQuery, there's also Mika Tuupola's Jeditable plugin. According to the comments Mika intends to add .live() support soon, but until he does this should work just fine.
"Why did you choose Editable over Jeditable if Jeditable is more popular?"
In this specific case I didn't want to submit anything back to the server on edit, and it seemed like Jeditable was all about saving back to a data store somewhere. Editable just seemed simpler for what I was working on at the moment.
Happy programming!
Now that we have that out of the way, another one of jQuery's really great features that mimics why Wordpress is so great - all of the user contributed plugins, whether they're efficient or not, the fact that someone else has already created a solution for you, and is willing to share it with you (most likely for free) is awesome. One such type of plugin that I was looking for today, and knew I'd want to eventually use at some point, was an edit-in-place, or inline edit plugin - something that allowed me to edit content from within the DOM, such as in a DIV container (not in a textbox - or at least not initially rendered in one) dynamically.
In my case today, I was experimenting with dynamically creating elements that might need to be edited after being created. My first gut-instinct thought? Yup, you guessed it: .live(). I replaced the call to .bind() in the plugin's source and voilà! Nothing. It didn't work. The jQuery code in the plugin had been optimized beyond a point where I was comfortable to modify it further and understand what I was doing in order to make it work (which was kind of pathetic, but I digress).
So, how could I solve my little problem? Quite easily - I wasn't really thinking.
The call to instantiate an event on targeted objects after the DOM is ready is something like this:
I was scratching my head, trying to figure out if I'd have to uninstantiate all elements and reinstantiate the event bubbling throughout the DOM each time I created a dynamically created element. I was over-thinking it. Instead, all I had to do was create the event on that newly created element on the element's creation itself. Ooh, hard. I know, I know - I'm brilliant!
jQuery v1.4+ example code:
Easy and simple.
You can see how to further use and extend Arash Karimzadeh's jQuery Editable plugin, or if you'd prefer a more popular plugin for inline editing with jQuery, there's also Mika Tuupola's Jeditable plugin. According to the comments Mika intends to add .live() support soon, but until he does this should work just fine.
"Why did you choose Editable over Jeditable if Jeditable is more popular?"
In this specific case I didn't want to submit anything back to the server on edit, and it seemed like Jeditable was all about saving back to a data store somewhere. Editable just seemed simpler for what I was working on at the moment.
Happy programming!
Adobe AIR 2.0
Friday, March 19. 2010
As I was redesigning the interface to my "RefStat" (Reference Statistics Tracking) application, I came across a web page that claimed support for CSS3 in Adobe AIR, so immediately began creating the new interface using Safari 4 as a testing ground for my UI. After almost completing it, it was brought to my attention that this would not be available until version 2 of AIR was released. I am still using v1.5.x. Not awesome.
The good news however, is that there is a public release of AIR 2.0 Beta 2 for download, which gives me hope that the stable release is not too far away. Since I am also waiting on an update to the tooltip library I am using, this wait hopefully won't be too long after I am ready for it.
New features in Adobe AIR 2.0 that I am looking forward to:
Features planned for my redesigned UI (v2):
I might add a graph rendering to display current stats for the day too as I've been asked for it, but it wasn't meant to view reports, only create the data used for them. This seems less important to me than the user interface and code updates though (template/skinning was so users could switch between horizontal and vertical displays depending on their desktop icon arrangements or resolution).
Current mockup (without tooltip) rendered by Safari 4.0.5 for Windows.
I never claimed to be a UI designer. ;) The handles on the left/right sides are "grip" or handle areas in order to move the application. The resize/change template button is not yet implemented in this template (though all controls would be in the orange group). The notepad is the new feedback button (in place of the "T|" from before), and of course minimize and close. The image shows a hover state, not a mouse-down state. The colors are meant to be similar to our library's "official" color (#621E27 or close variations).
The good news however, is that there is a public release of AIR 2.0 Beta 2 for download, which gives me hope that the stable release is not too far away. Since I am also waiting on an update to the tooltip library I am using, this wait hopefully won't be too long after I am ready for it.
New features in Adobe AIR 2.0 that I am looking forward to:
- UDP packet support (interfacing with some game servers)
- TCP/open socket listening (acts as server)
- ~30% more efficient than AIR 1.x
- WebKit version updated to Safari 4.0.3 equivalent
- (Better?) Printer support
Features planned for my redesigned UI (v2):
- 8+ more buttons? 800x600 or less resolution? No problem anymore! The buttons wrap and the app scales.
- Templates/Skinning support
- No more color coded buttons (in default templates)
- Redone UI in my template won't have the hover bug for tooltips (mouseout from inside AIR app, thinks you're still hovering)...AIR 2.0 might fix this on its own.
- On-the-fly template switching
- Refactored object class
I might add a graph rendering to display current stats for the day too as I've been asked for it, but it wasn't meant to view reports, only create the data used for them. This seems less important to me than the user interface and code updates though (template/skinning was so users could switch between horizontal and vertical displays depending on their desktop icon arrangements or resolution).
Current mockup (without tooltip) rendered by Safari 4.0.5 for Windows.
Reference Statistics Tracker for SSPL v2
I never claimed to be a UI designer. ;) The handles on the left/right sides are "grip" or handle areas in order to move the application. The resize/change template button is not yet implemented in this template (though all controls would be in the orange group). The notepad is the new feedback button (in place of the "T|" from before), and of course minimize and close. The image shows a hover state, not a mouse-down state. The colors are meant to be similar to our library's "official" color (#621E27 or close variations).
