Build Optimization - Getting list of big images/files shipped during release

Large images in a webpage is the one of the biggest contributor to performance problems in most web pages. Most of such issues are catched during performance testing when you see large images taking long time to load and also increasing the overall size of page.

Small changes can have a big impact

As a build person I always felt the best time to catch such issues is during build time. A designer might make a mistake of committing large images in Source Control but if we have a mechanism to catch it in next CI build will be great. To make it possible I needed a script which I can hookup with my build or may be jenkins job to take care of this task for me. I used the Linux "find" command with ANT .....wallahhh !!! 

I have list of all JPG/PNG/GIF images which have size greater then 100 kb.

 <target name="get-length">

  <echo file="mail.txt" append="false">List of JPG images :-${line.separator}</echo>
  <exec executable="sh" append="true" output="mail.txt">
     <arg value="-c" />
     <arg value="find /deploy/HEAD -path '*/dist/*' -prune -o -iname '*.jpg' -size +100k -exec ls -lh {} \; | awk '{print $5,$9}'" />
  </exec>
  <echo file="mail.txt" append="true">List of PNG images :-${line.separator}</echo>
  <exec executable="sh" append="true" output="mail.txt">
     <arg value="-c" />
     <arg value="find /deploy/HEAD -path '*/dist/*' -prune -o -iname '*.png' -size +100k -exec ls -lh {} \; | awk '{print $5,$9}'" />
  </exec>
  <echo file="mail.txt" append="true">List of GIF images :-${line.separator}</echo>
  <exec executable="sh" append="true" output="mail.txt">
     <arg value="-c" />
     <arg value="find /deploy/HEAD -path '*/dist/*' -prune -o -iname '*.gif' -size +100k -exec ls -lh {} \; | awk '{print $5,$9}'" />
  </exec>
 </target> 


Hope this helps.

Power of ANT

I am in love with ANT & I seriously mean it. Whenever I think of automation, there is no way I can ignore it. There is so much you can do to automate your daily tasks. Deployment of builds across multiple platforms, restart services, link alive checks, reports generation and even running mysql queries. I consider myself very lucky as whenever I came across any issues I always found the solution using various ANT commands. I will be sharing few important utilities I created using ANT in future blogs but this is something I implemented today.

The task in hand was that while deploying builds manually on stage bed or production sometimes the configuration files were overwritten by the person deploying it. Although the occurrence of it was very low but the impact was huge which can result in downtime of the application. The request was to rename all occurrence of some configuration xml/properties files to be changed to FileName.xml.template/*FileName.properties.template so that config files doesn't get overwritten. I thought it was worth sharing the ant code.

 <move todir="${basedir}/SourceDirectory" includeemptydirs="false">  
       <fileset dir="${basedir}/SourceDirectory">  
         <include name="**/dbconfig.xml"/>  
         <include name="**/webconfig*.xml"/>   
         <include name="**/validip.properties"/>  
         <include name="**/web.xml"/>  
         <include name="**/ServiceSetting.xml"/>  
         <include name="**/logsettigs.properties"/>  
       </fileset>  
       <mapper type="glob" from="*" to="*.template"/>  
  </move>  

Simple commands which processesed around 53,000 files in CVS checkout directory and changed total of approx 530 files in just 1 Sec on a CentOS machine.

One more interesting thing which I implemented was using the macrodef. This is a part of advance ant technique which is supported after ant 1.6 and later. I was working on creating template based build.xml for few projects which used common code base to create multiple deployments. Consider a lib folder needs to be deployed over 10-20 different locations. A simplified macrodef use can really help over come this and reduce reduntant code use.

A simple example :-

 <macrodef name="copylibs">  
  <attribute name="todir" />  
    <sequential>  
      <copy todir="@{todir}" flatten="true" overwrite="true" includeEmptyDirs="no" failonerror="false">  
       <fileset dir="${sourcedirectorytocopyfilesfrom}">  
         <include name="lib/*.jar"/>  
       </fileset>  
      </copy>  
    </sequential>  
 </macrodef>  
 <copylibs todir="${destinationdirectory}/project1/WEB-INF/lib" />  
 <copylibs todir="${destinationdirectory}/project2/WEB-INF/lib" />  
 <copylibs todir="${destinationdirectory}/project3/WEB-INF/lib" />  
 <copylibs todir="${destinationdirectory}/project4/WEB-INF/lib" />  
 <copylibs todir="${destinationdirectory}/project5/WEB-INF/lib" />  

In this way you can overcome some of daily problems faced. In case you are facing some issues similar to what I described or if there something you want to implement and need a solution or guidance for the same, do let me know. I will try to give you the best possible solution.