CSV files with Byte Over Marks

I've been working with both the DATA-INTO and the free CSV utility that Scott Klement wrote. I have a couple of CSV files that I generate daily out of our payroll system. If I try and do a simple client access data transfer I end up with some weird characters in the first column of the first record. I also get similar results when I use the DATA-INTO and the CSV utility from Scott Klement. If I run the program in debug mode I see that the first field of the first column is padded with what looks like this.


It was suggested to me that the issue could be that the CCSID is not UTF-8. It was also mentioned that I may be dealing with files that have BOM (Byte over mark) in them. I'm not sure if I'm doing this correctly, but if I do a WRKLNK, locate the file, and check the properties of the file on the IFS I see the coded character set ID is 1252. I'm not sure if that means the file is UTF-8 or not. How do I get around the issue of the BOM? Any help would be greatly appreciated. What I have been doing in the mean time is downloading the file as an Excel file, opening it up in Excel, and then saving it as a CSV file. Doing that drops the BOM, but that seems like a lot of extra work.

Michael Soucy
Sr. Programmer / Analyst
Community Health and Counseling Services
(207) 922-4837

****** CONFIDENTIALITY NOTICE ******

The documents accompanying this email contain confidential information that may be legally privileged and protected by federal and state law. This information is intended for use only by the entity or individual to whom it is addressed. The authorized recipient is obligated to maintain the information in a safe, secure, and confidential manner. The authorized recipient is prohibited from using this information for purposes other than intended, prohibited from disclosing this information to any other party unless required to do so by law or regulation, and is required to destroy the information after its stated need has been fulfilled.

If you are in possession of this protected health information, and are not the intended recipient, you are hereby notified that any improper disclosure, copying, or distribution of the contents of this information is strictly prohibited. Please notify the owner of this information immediately and arrange for its return or destruction. Thank You.