For one of my project I needed to remove certain footer / tail-end lines from the large-file to make it clean as it was containing some summary of records within the file. Because if I do that it can be directly loaded into database by LOAD DATA command.
Usually one can read file into an array using PHP's file function but that's okay when size of file is in few hundred KB, if your file-size is running in to few MB to few hundred MB we need to find and use some efficient way to play with them. Because with large file reading as an array we can (and certainly will) run into memory outage issue.
To remove lines from the end of file you can use following snippet of code:
// Function to truncate last N lines from the file function truncate_last_n_lines_of_file($file, $lines_to_remove, $chunk = 1024) { // Open file in read+write mode $handle = @fopen($file, "a+"); $lines_found = array(); // Check if it's a valid file handle if($handle) { // size of file $max_length = $file_size = filesize($file); if(intval($file_size) == PHP_INT_MAX) { $max_length = PHP_INT_MAX; } // loop through file as long as we are not done with truncating required files for($length = 0; $length < $max_length; $length += $chunk) { if( ($max_length - $length) > $chunk) { $seek_size = $chunk; } else { $seek_size = $max_length - $length; } // read data in chunk fseek($handle, ($length + $seek_size) * -1, SEEK_END); $data = fread($handle, $seek_size) . $data; // Loop thorugh chunk to see if we are done with truncate for($i = $chunk; $i > 0; $i--) { if($data[$i-1] == "\n") { $lines_found[] = $i; } if(count($lines_found) == $lines_to_remove+1) { ftruncate($handle, ($max_length) - ($seek_size - $lines_found[$lines_to_remove])); return true; } } fclose($handle); } return false; } }
