htmlentities() in PHP is Your Friend
Friday, October 14. 2011
After constantly badgering a certain library calendar vendor over 2 years to fix his software's RSS feed charset issues. Personally I don't think getting raw text describing times in the form of "6:30–8:30 p.m." is all that valuable...that's just a single example. The calendar website declared no charset information. I have no idea what charset the database is in, and the RSS feed was declared as ISO-8859-1. Our website, database, and everything else was declared as UTF-8, not that it really mattered though since the raw incoming text from the RSS feed was all garbled to begin with.
Every once and awhile I'd randomly try to find an answer to the problem. I've been through using all sorts of different algorithms to solve the problem. None of them seemed to work, until one day I saw someone mention on StackOverflow (unfortunately I've lost the link) that he tried using htmlentities() to solve his problem and it worked. I thought, "It couldn't be that simple..." However, I had nothing to lose and tested it. It worked. (What???) I still don't know why or how htmlentities managed to run a translation table on the garbled input to output the appropriate values, but I'm happy! Even my attempts at REGEX were unsuccessful, though I probably was just unable to find ALL the right bit-level character code sequences needed. Apparently the translation table that htmlentities uses is pretty darn thorough! Thanks, PHP team!
Okay, so that was the first use of htmlentities(). The second one?
I realized I overlooked a severe security hole in my forms. When users did not provide correct details in their forms, I was simply reinserting the values they provided back in to the HTML form's VALUE tag (or in the case of a textarea, just rendering the value between the tags). For some reason this didn't strike me as being severely stupid at the time. I don't know why. I guess the "never reprint what your users submit to you" only made me think of "back to the DOM" - but only outside form elements. Who knows why. This let someone who actually put some (minimal) thought into it to run whatever PHP code they wanted simply by submitting a form without all required data. Escape the form element by using a standard HTML closing tag, then start writing the PHP. If you wanted valid HTML, just make sure to also include a dummy HTML input or textarea field once done. Simple. (Note: I am also in the process of re-examining CHMOD values of files and folders.)
When I went back to "fix" my stupidity, I also initially thought of using PHP's filter functions. Although they worked, they also would sometimes (depending on user input) remove certain characters. Like a bolt of lightning (while I was eating lunch) it came to me. I just used htmlentities(), why not just use it again? ...so I did. Now my forms are a bit more protected and our RSS feed is no longer displaying obnoxious characters to visitors due to the encoding mishaps of an external developer.
Sometimes PHP's little gems are so awesome...
Every once and awhile I'd randomly try to find an answer to the problem. I've been through using all sorts of different algorithms to solve the problem. None of them seemed to work, until one day I saw someone mention on StackOverflow (unfortunately I've lost the link) that he tried using htmlentities() to solve his problem and it worked. I thought, "It couldn't be that simple..." However, I had nothing to lose and tested it. It worked. (What???) I still don't know why or how htmlentities managed to run a translation table on the garbled input to output the appropriate values, but I'm happy! Even my attempts at REGEX were unsuccessful, though I probably was just unable to find ALL the right bit-level character code sequences needed. Apparently the translation table that htmlentities uses is pretty darn thorough! Thanks, PHP team!
Okay, so that was the first use of htmlentities(). The second one?
I realized I overlooked a severe security hole in my forms. When users did not provide correct details in their forms, I was simply reinserting the values they provided back in to the HTML form's VALUE tag (or in the case of a textarea, just rendering the value between the tags). For some reason this didn't strike me as being severely stupid at the time. I don't know why. I guess the "never reprint what your users submit to you" only made me think of "back to the DOM" - but only outside form elements. Who knows why. This let someone who actually put some (minimal) thought into it to run whatever PHP code they wanted simply by submitting a form without all required data. Escape the form element by using a standard HTML closing tag, then start writing the PHP. If you wanted valid HTML, just make sure to also include a dummy HTML input or textarea field once done. Simple. (Note: I am also in the process of re-examining CHMOD values of files and folders.)
When I went back to "fix" my stupidity, I also initially thought of using PHP's filter functions. Although they worked, they also would sometimes (depending on user input) remove certain characters. Like a bolt of lightning (while I was eating lunch) it came to me. I just used htmlentities(), why not just use it again? ...so I did. Now my forms are a bit more protected and our RSS feed is no longer displaying obnoxious characters to visitors due to the encoding mishaps of an external developer.
Sometimes PHP's little gems are so awesome...
Finding all application Controllers in CakePHP (v1.3.x)
Tuesday, September 28. 2010
At some point in time you may find the need to get a list of all of the controllers within your application. It's actually quite simple so long as you don't need controllers from any plugins.
Place this code in any of your controllers and view it from the web (for instance, from the Users Controller): http://www.example.com/users/listControllers
You'll see something similar to the following:
The App::objects() method returns an array of objects of the given type, such as: 'model', 'controller', 'helper', or 'plugin' - it also accepts other parameters, such as "path" in case you do eventually need to check controllers for your plugins.
The array_diff() is there as a much simpler method to remove the AppController and PagesController from the returned results as they would most likely exist in your application regardless and aren't normally something you'd need to worry about with ACL as there are other means within Cake to handle access to them. You'll notice that array_diff doesn't return an array starting at index of 0 since it removes keys from the original array too. That shouldn't be a problem, but if it is you can always use a loop instead of array_diff() and just unset or splice the matching values.
I used the CakePHP convenience method of pr() (print_r surrounded by PRE tags), and a die() simply to show the returned results and print them to the screen. You'd probably prefer to double underscore the function name (__listControllers()) to make it a private method to the class (rather than publicly viewable via the web), and change pr() to return.
What's the purpose of this?
- Maybe you'd want to create a web interface for ACL and need to know which controllers to give/deny access to/from
- Maybe you want to create a navigation menu based on your controllers
...maybe you can think of something that I can't. :)
Alter to your own tastes. This is only a starting point.
Other useful links (check version compatibility in these resources):
Quick dessert: List all controllers of a CakePHP application
Automatically load all controllers and actions into ACO tables for ACL with a CakePHP Task
How to list all controllers
Place this code in any of your controllers and view it from the web (for instance, from the Users Controller): http://www.example.com/users/listControllers
You'll see something similar to the following:
The App::objects() method returns an array of objects of the given type, such as: 'model', 'controller', 'helper', or 'plugin' - it also accepts other parameters, such as "path" in case you do eventually need to check controllers for your plugins.
The array_diff() is there as a much simpler method to remove the AppController and PagesController from the returned results as they would most likely exist in your application regardless and aren't normally something you'd need to worry about with ACL as there are other means within Cake to handle access to them. You'll notice that array_diff doesn't return an array starting at index of 0 since it removes keys from the original array too. That shouldn't be a problem, but if it is you can always use a loop instead of array_diff() and just unset or splice the matching values.
I used the CakePHP convenience method of pr() (print_r surrounded by PRE tags), and a die() simply to show the returned results and print them to the screen. You'd probably prefer to double underscore the function name (__listControllers()) to make it a private method to the class (rather than publicly viewable via the web), and change pr() to return.
What's the purpose of this?
- Maybe you'd want to create a web interface for ACL and need to know which controllers to give/deny access to/from
- Maybe you want to create a navigation menu based on your controllers
...maybe you can think of something that I can't. :)
Alter to your own tastes. This is only a starting point.
Other useful links (check version compatibility in these resources):
Quick dessert: List all controllers of a CakePHP application
Automatically load all controllers and actions into ACO tables for ACL with a CakePHP Task
How to list all controllers
Website Render Time with PHP >= 5.1
Tuesday, March 2. 2010
In CakePHP v1.3.x, there will no longer be an automatic dump of tons of information (probably because the DebugKit plugin can handle all of that information for us) when in debug mode. However, in my non-CakePHP websites, and even in my Cake websites, I have become accustomed to viewing the source to see a full render time for the page in an HTML comment. Thanks to the help of "jnay" over at CakeQs.org, I've found out about a new (as of PHP 5.1) $_SERVER variable argument called "REQUEST_TIME". It holds the UNIX timestamp of the initiation of the call.
By using this variable, along with the optional boolean argument for microtime (as of PHP 5.0.0), we can determine the HTML render time of our application with the following calculation in either our footer view/template, or controller (preferably in the last possible iteration of your project's render):
The echo is optional and used only as an example here; it simply depends on how you use the calculation in your script(s)/framework(s). This is much easier than starting the timer in the header and calculating the difference in the footer like I had to do in the PHP 4 days!
Remember, this is only for trivial estimations and comparisons. If you need to thoroughly evaluate your script(s) for time efficiency you should be using something else, such as Xdebug and a form of CacheGrind for your choice of OS.
By using this variable, along with the optional boolean argument for microtime (as of PHP 5.0.0), we can determine the HTML render time of our application with the following calculation in either our footer view/template, or controller (preferably in the last possible iteration of your project's render):
The echo is optional and used only as an example here; it simply depends on how you use the calculation in your script(s)/framework(s). This is much easier than starting the timer in the header and calculating the difference in the footer like I had to do in the PHP 4 days!
Remember, this is only for trivial estimations and comparisons. If you need to thoroughly evaluate your script(s) for time efficiency you should be using something else, such as Xdebug and a form of CacheGrind for your choice of OS.
Server Load, Take 2
Friday, January 29. 2010
So I worked on the CakePHP plugin for Prowl a bit more, and in doing so, I created shell-based access to the plugin. I can now send Prowl messages from the console (i.e.: I can run PHP via cron to check on things and send myself the results, if necessary). A shell task I created was for server load, as a followup to my previous post. In working this out, I've modified the output for CakePHP's Shell::out() method so that it auto-wraps and indents a bit more nicely (my ticket in the LighthouseApp is here). Time for some output examples:
Issuing the Load Alert help call: cake prowl load_alert help
Sending a message is as simple as: cake prowl msg [apikey] ... it can get as advanced as cake prowl msg [apikey] [priority] [application] [event] [description]
The output for running the load alert command?
If it's above my arbitrarily set values, it'll send a different warning depending on the level to my Prowl-enabled device. By sending a "-silent 1" parameter to the command message, output is disabled (perfect for cron).
Please note that I don't have any styling for shell output (something I'll have to fix) so the pasted character spacing and newlines won't work as expected in my example output.
Issuing the Load Alert help call: cake prowl load_alert help
Sending a message is as simple as: cake prowl msg [apikey] ... it can get as advanced as cake prowl msg [apikey] [priority] [application] [event] [description]
The output for running the load alert command?
If it's above my arbitrarily set values, it'll send a different warning depending on the level to my Prowl-enabled device. By sending a "-silent 1" parameter to the command message, output is disabled (perfect for cron).
Please note that I don't have any styling for shell output (something I'll have to fix) so the pasted character spacing and newlines won't work as expected in my example output.
Creating an array of date time values via PHP
Thursday, October 22. 2009
I was working on my Reference Statistics Tracker for work, and ran in to a snag.
I am using eZ Components' ezGraph class to create image-based graphs on the fly. It's fast, efficient, and it looks nice - it can even render in SVG (I'll be using JPG or PNG...thanks IE). For my particular needs, I wanted the exported dataset to have a specific formatting to the keys, to represent the different values in my graphs. For instance, let's say I have the following stats from 100 users in our company, and the Operating System they use. This is purely fictitious:
So now I just had to pull the values from the database in the format I wanted. Well my keys in this case were dates, or times (span of days, or hours in a day, or month in a year, etc...). The values were still numeric values in this case. So, we're looking at something like:
The PHP function of range() just didn't cut it. Although I could have converted the datetime values to timestamp values (either via SQL or PHP), and used the 3rd parameter of the function to add the specific number of seconds needed, I was weary of doing that as I remember reading about issues that could cause (adding seconds to a timestamp value calculate future time). Besides, I'd still have to use array_walk() or array_map() to format my array's keys the way I wanted; that left yet another function, and a line of code for the call to array_walk/map. (My host does not yet have PHP 5.3.x, which will allow for lambda functions in array_map.) So, I decided to create my own little function, and figured I'd post it here in case (1)someone could use it and it'd help them out, or (2)someone thinks there's a more efficient way of doing this and wants to give feedback.
This could be useful for similar graphing/charting code, I don't think it's specific to eZ Components'. Perhaps it could be useful for something else that I can't think of. Have fun with it.
I am using eZ Components' ezGraph class to create image-based graphs on the fly. It's fast, efficient, and it looks nice - it can even render in SVG (I'll be using JPG or PNG...thanks IE). For my particular needs, I wanted the exported dataset to have a specific formatting to the keys, to represent the different values in my graphs. For instance, let's say I have the following stats from 100 users in our company, and the Operating System they use. This is purely fictitious:
array('Windows' => 90, 'OSX' => 6, 'Other' => 4);That specific array is crafted perfectly in the format that works well with eZ Components' Graphing class, for a simple bar chart, or a pie chart.So now I just had to pull the values from the database in the format I wanted. Well my keys in this case were dates, or times (span of days, or hours in a day, or month in a year, etc...). The values were still numeric values in this case. So, we're looking at something like:
array('HH:MM:SS' => 10);However, no matter how hard I tried, if there were no values inserted in to the database for a specific time, I could not return a value of null, or zero - even with an outer join. My next thought was to use something like array_merge() on the resulting dataset to fix it.The PHP function of range() just didn't cut it. Although I could have converted the datetime values to timestamp values (either via SQL or PHP), and used the 3rd parameter of the function to add the specific number of seconds needed, I was weary of doing that as I remember reading about issues that could cause (adding seconds to a timestamp value calculate future time). Besides, I'd still have to use array_walk() or array_map() to format my array's keys the way I wanted; that left yet another function, and a line of code for the call to array_walk/map. (My host does not yet have PHP 5.3.x, which will allow for lambda functions in array_map.) So, I decided to create my own little function, and figured I'd post it here in case (1)someone could use it and it'd help them out, or (2)someone thinks there's a more efficient way of doing this and wants to give feedback.
This could be useful for similar graphing/charting code, I don't think it's specific to eZ Components'. Perhaps it could be useful for something else that I can't think of. Have fun with it.
