File Manipulation

-=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- (c) WidthPadding Industries 1987 0\|605\|0 -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=- -=+=-
Socoder -> Blitz -> File Manipulation
Mon, 06 Dec 2010, 03:34
mole	Hi, Long time no see. I still lurk occasionally but I haven't posted for a while and I certianly haven't coded for a while. Hence, I am rusty. Therein lies my dilemna: I have a task I wish to complete, and I believe that a little Blitz program will make this easier.... If only I could properly remember how to program, that is. Anyway, I'll start off with my first problem. I am trying to read a column of data from a text file. The first line of the column is a text string and the lines beneath are floats. I am trying to commit this data to 2 arrays: 'FirmName$(a)' for the string, and all of the rest of the floats into 'DataSet1#(b)'. Here if my code so far. Unfortunately, I have got stuck quite early on in the sense that my debug reporting suggests that it can't see any data in the file... AppTitle "Correlation Coefficient Analyser" Graphics 640,480,32 SetBuffer BackBuffer() ;Calcualte correlation coefficient between each of ;of the firms uaing data from the data files Global FirmNumber = 0;Number used to iterate through the names of the firms Global OutputCounter = 30 ;For displaying the data in a column Dim DataSet1#(324) Dim FirmName$(2) Correl("CorrelData.dat") While Not KeyHit(1) Text 10,10,"Processing correlation data..." ;Correl("CorrelData.dat") Text 10,20,FirmName$(0) ; For j = 0 To 322 ; Text 10,OutputCounter,DataSet1#(j) ; OutputCounter = OutputCounter + 10 ; Next Flip Cls Wend Function Correl(DataFile$) CorrelData = ReadFile(DataFile$) If CorrelData = 1 Then DebugLog("Opened File Successfully") Else DebugLog("No data found!") For i = 0 To 10 If i = 0 NameOfFirm$ = ReadString(CorrelData) FirmName$(FirmNumber) = NameOfFirm$ Else Read1# = ReadFloat(CorrelData) DataSet1#(i) = Read1# EndIf Next End Function --v Any help, links, tips would be super cool. I will no doubt return quickly after this problem is solved with another problem, as my program develops in size and sophistication. Thanks in advance!
Mon, 06 Dec 2010, 03:55
Jayenkai	Is the "Float" stored AS a float, or is it a line of text? (as in, if you open the .dat in notepad, is it readable, or gobbledegook) If so, ReadTxt$=ReadLine(CorrelData): Read1#=float(ReadTxt$) -=-=- ''Load, Next List!''
Mon, 06 Dec 2010, 08:11
mole	The data in the file is readable. I will upload it for reference. I managed get the program to output the data as I expect it to; the data is being stored as I want. However, that means we now have the next step. By the end, the file will have anything up to 100 columns of data. I need to be able to store all that data in arrays (I can't quite remember but can you have arrays such as 'Dim ArrayName(5,5)'?). When all the data is stored, I hope to be able to calculate the correlation coefficients between each of the columns of data. That is something I will consider more deeply when I get to it; at the moment I am just trying to get all the data into array ready for processing. What I can't think of how to do, is the parse/read the file of data and get each chunk of data into its appropriate place in the array. that is where I need help. I never was good at file manipulation Here is the latest code: AppTitle "Correlation Coefficient Analyser" Graphics 640,480,32 SetBuffer BackBuffer() ;Calcualte correlation coefficient between each of ;of the firms uaing data from the data files Global FirmNumber = 0;Number used to iterate through the names of the firms Global OutputCounter = 34 ;For displaying the data in a column Global NameOfFirm$ Dim DataSet1#(324) Dim FirmName$(2) Correl("CorrelData.dat") While Not KeyHit(1) Text 10,10,"Processing correlation data..." ;Correl("CorrelData.dat") Text 10,22,FirmName$(0) For j = 0 To 323 Text 10,OutputCounter,DataSet1#(j) OutputCounter = OutputCounter + 12 Next OutputCounter = 34 Flip Cls Wend Function Correl(DataFile$) CorrelData = ReadFile(DataFile$) If CorrelData = 1 Then DebugLog("Opened File Successfully") Else DebugLog("No data found!") For i = 0 To 324 If i = 0 NameOfFirm$ = ReadLine(CorrelData) FirmName$(FirmNumber) = NameOfFirm$ Else Read1# = ReadLine(CorrelData) DataSet1#(i-1) = Read1# EndIf Next End Function --v Correlation Data File Note that I had to change the file extension to the less cool .txt in order to upload it.
Mon, 06 Dec 2010, 17:57
Andy_A	Here's what it sounds like you want. ;Calcualte correlation coefficient between each of ;of the firms uaing data from the data files Global FirmNumber = 0;Number used to iterate through the names of the firms Global OutputCounter = 34 ;For displaying the data in a column Global NameOfFirm$ Dim DataSet1#(2,324) Dim FirmName$(2) Graphics 800,600,32,2 SetBuffer BackBuffer() recordsRead = Correl("CorrelData.dat") Cls Text 5,10, "Records Processed: "+recordsRead y1 = 30 y2 = 50 y3 = 70 For lines = 0 To 59 Step 10 xPos = 80 For record = 0 To 9 Text 5,y2, FirmName$(0) Text 5,y3, FirmName$(1) Text xPos, y1, "Rec"+(lines + record) Text xPos, y2, DataSet1#(0, (lines+record)) Text xPos, y3, DataSet1#(1, (lines+record)) xPos = xPos + 70 Next y1 = y1 + 90 y2 = y2 + 90 y3 = y3 + 90 Next Flip While Not(KeyHit(1)) Wend End Function Correl(DataFile$) tab$ = Chr$(9) ;I noticed that this is a TAB delimited file CorrelData = ReadFile(DataFile$) If CorrelData <> 0 Then ;file opened successfully - proceed lineCounter = 0 While Not(Eof(CorrelData)) ;read in line of data and use program logic to determine what to do with it lineInput$ = ReadLine(CorrelData) If lineCounter = 0 Then ;this is the first record, it must be the record with Firm Name's For i = 0 To 999 ;1000 is a very high max to make sure to read all of the firm names nextFirm$ = Word(lineInput$, i+1, tab$) ;using a TAB char as the delimiter ;Exit the For..Next loop if no more Firm Names If nextFirm$ = "" Then Exit FirmName$(i) = nextFirm$ Next Else For i = 0 To 999 ;again with improbable max to make sure we read all of the data ;get just 1 value from current record dataPoint$ = Word(lineInput$, i+1, tab$) If dataPoint$ = "" Then Exit DataSet1#(i, lineCounter - 1) = Float(dataPoint$) Next End If lineCounter = lineCounter + 1 Wend CloseFile(CorrelData) ;Don't forget to close opened files Else RuntimeError "File: "+DataFile$+" - Failed To Load!" End If Return (lineCounter-1) End Function Function Word$(string2Chk$, n, delimiter$=" ") Local count%, findDelimiter%, position%, current$ count = 0 findDelimiter = 0 position = 1 If n > 0 Then string2Chk$ = Trim(string2Chk$) Repeat findDelimiter% = Instr(string2Chk$,delimiter$,position) If findDelimiter <> 0 Then current$ = Mid$(string2Chk$,position,findDelimiter-position) count = count + 1 position = findDelimiter + 1 If count = n Then findDelimiter = 0 End If Until findDelimiter = 0 If (count < n) And (position <= Len(string2Chk$)) Then current$ = Mid$(string2Chk$,position, Len(string2Chk$) - position+1) count = count + 1 If count < n Then current$ = "" End If End If Return current$ End Function --v
Wed, 08 Dec 2010, 06:19
mole	I have spend the last day and a half working on your code, Andy, and fine tuning it/modifying it to my needs. Your structure has proved invaluable in allowing me to progress, so a big thanks for that. The latest I have done so far is make it super easy to change the number of firms in the data file without making the program have a wobbly, introduce a correlation coefficient calculator function, and implement it in a manner such that it calculates the coefficients between all the firms in the data file. Calculating the correlation coefficient between the data from the file is exactly what I wanted - and it works! This makes me happy. So now that the program is robust enough for my needs, all I need to do now is get data on another 95 firms or so!! I should be able to make any minor modifications to the program if the need arises, but I'll be back if there is something that stumps me Many thanks for your help, Jay and Andy. It is most appreciated.
Wed, 08 Dec 2010, 06:35
Jayenkai	Sorry, I completely forget to get back to this topic! Glad you sorta sorted it! -=-=- ''Load, Next List!''
Wed, 08 Dec 2010, 08:44
Andy_A	Glad it all worked out for you. Getting the data into arrays was the toughest part (and it wasn't all that hard).