Make Scar find/read/save text

[Bixby Sayz]

if (GetColor(945,600) = water) then   //if there is water instead of an island,
     Continue;                          //skip to next coordinate

Why doesn't this skip the For loop it's currently in?

Continue will loop back up to top of the current loop and continue with the next iteration (if currently on yi=0 then start at top of loop with yi=1)

 

Break will break out of the current loop. In this case break out of the inner loop but carry on with the code immediately following the inner for loop (or in this case loop back up to the top of the outer for loop)

 

Is this what you are trying to do? Jumping into this topic late so may have misread what you are trying to do.

[MarkD]

xi and yi are coordinates. If the if-statement is true, it has to skip that coordinate (so restart that loop with yi+1).

I guess that's the first option you mention, so the Continue statement seemed the correct option.

[Bixby Sayz]

Looks like it should work. Only thing I can suggest is putting a WriteLn of getcolor output to see what it is really reading, and double check value of water.

[MarkD]

Water appears to be non-constant... Grmbl

 

When I insert a WriteLn(GetColor(945,600)) I get the following output:

9892351
11564307 *
2042973
1805177
1805177 *
1805177 *
1805177
1805177
9432043
2042973
2042973
2042973
1281909
2042973
2042973
2042973

 

The 2nd, 5th and 6nd should be water.

Only the 2nd one appears to be close to water. I'm trying to figure out why it looks like this now...

 

EDIT:

Perhaps my mouse cursor is messing things up. The check is located after the mouse movement. Since it's a pretty fancy mouse cursor (more then 20 different colors in it I guess), I should've ruled that out beforehand... I'll keep you guys posted.

Edited July 28, 2012 by MarkD

[MarkD]

New test:

9892351
11564307 *
2042973
1297072
11564307 *
11564307 *
2042973
1297072
9432043
2042973
2042973
2042973
1281909
2042973
2042973
2042973

 

Seems stabilized

 

Now a test with coordinates displayed each time the island is clicked:

9892351
0
0
11564307
2042973
0
2
1297072
0
3
11564307
11564307
2042973
1
2
1297072
1
3
9432043
2
0
2042973
2
1
2042973
2
2
2042973
2
3
1281909
3
0
2042973
3
1
2042973
3
2
2042973
3
3
Done

 

The water doesn't get clicked! So, for some reason, it also gets the color of the mouse cursor...

At least it works now

[LordJashin]

Yeah sorry forgot to comment on Continue;. Break; exits the current loop, Exit; gets out of everything and exits the procedure/function.

 

I don't understand what you are doing exactly but.

 

It is very hard to get a Reflection script working, at that level if it incorporates factors such as ZOOM. GetColor, GetTextAt, Get anything from a certain point means there has to be something at that point. Even if you did get it working it would be very plain and low level. Be like using DTM's but probably less accurate?

 

Anyway, here's what I would do. If you want the easy route make a DTM Tools->DTM Editor for w/e you are trying to find. Or use bitmaps Tools->Pick Bitmap.

 

Bitmap Example:

 

[sCAR]

program New;

 

var

bmp, bmp1: TSCARBitmap;

Client: TSCARWindowClient;

ClientArea: TBox;

x, y: Integer;

 

procedure SetupBitmaps;

begin

bmp := TSCARBitmap.Create('');

bmp1 := TSCARBitmap.Create('');

bmp.LoadFromBmp('C:\bmp.bmp');

bmp1.LoadFromBmp('C:\bmp1.bmp');

end;

 

procedure ScriptTerminate;

begin

bmp.Free;

bmp1.Free;

Client.Free;

end;

 

procedure SetupClient;

begin

Client := TSCARWindowClient(GetClient);

Client.Activate;

ClientArea := Client.ImageArea;

end;

 

procedure FindBitmap1;

var

i: Integer;

begin

wait(2000);

Repeat

wait(1000);

until (i = 10) or FindBitmap(x, y, bmp, 0, 0, ClientArea.X2, ClientArea.Y2);

if FindBitmap(x, y, bmp, 0, 0, ClientArea.X2, ClientArea.Y2) then

begin

// do stuff here

MoveMouse(x, y);

ClickMouse(x, y, mbLeft);

WriteLn('Found it');

end;

end;

 

 

begin

SetupBitmaps;

SetupClient;

FindBitmap1;

end.

 

[/sCAR]

 

If you really wanted to be accurate. Is the object 3D? Is it always the same colors all around? Either way a T2DIntegerArray makes all the difference. That with some tolerance. T2DIntegerArray can hold the RGB colors of each pixel (I think). I'm no expert but that I would consider the hard way to go unless you've extensively tried them out before. I have not too much, and they are harder to write. They are the best because if you can get all the colors from a box, and compare them with another box of colors. Take runescape for example, there is a minimap in a circle shape, SPS the runescape walking system takes a T2DIntegerArray of it, and compares it to the whole map that Runescape has posted in the game. Like you can view the map at a certain zoom and save each piece of it as pictures, then compare it to the minimap for walking. It has like 95%+ accuracy on walking.

Edited July 29, 2012 by LordJashin

[MarkD]

It has to find text once and read text once, but it's not the same text.

 

- Locate text (location is non-continuous, amount of texts is also non-continuous).

- Store location to not be clicked again.

- Click text location.

- If no text appears at given coordinates, skip next step.

- At given coordinates (so no need to find them first), there will be a few lines text. Read those lines. SAVE THOSE LINES!

- Click a button. ZOOM will automatically reset here, unfortunately.

- Repeat.

 

All the objects are 2D and the text it has to find is always surrounded for at least 2 pixels by a constant color. It doesn't have to read it, as long as the entire textbox is excluded from being clicked again in the future.

The lines it has to read (most will be as long as one word or a few numbers) do have a plain background. There are two colors of background, alternating each other per line. The start will always be the same color.

 

So, as soon as I get my OCR working and have something to zoom with, it should work. Let's save the saving till the moment the rest works.

At the moment I'm trying LordJashin's method, so I'm unsure at the moment which steps this could cover. I've never used DTM, so to prevent stupid mistakes I'm learning the basics of it before saying it will/won't work.

 

Last time I've played Runescape will be about 10 years ago, but I understand the map system hasn't changed much since. I suppose I could define something as textbox, and look for all places that look like a textbox. They are all the same size, it's only the text that differs. Is that what you're trying to say?

[LordJashin]

It has to find text once and read text once, but it's not the same text.

 

- Locate text (location is non-continuous, amount of texts is also non-continuous).

- Store location to not be clicked again.

- Click text location.

- If no text appears at given coordinates, skip next step.

- At given coordinates (so no need to find them first), there will be a few lines text. Read those lines. SAVE THOSE LINES!

- Click a button. ZOOM will automatically reset here, unfortunately.

- Repeat.

 

All the objects are 2D and the text it has to find is always surrounded for at least 2 pixels by a constant color. It doesn't have to read it, as long as the entire textbox is excluded from being clicked again in the future.

The lines it has to read (most will be as long as one word or a few numbers) do have a plain background. There are two colors of background, alternating each other per line. The start will always be the same color.

 

So, as soon as I get my OCR working and have something to zoom with, it should work. Let's save the saving till the moment the rest works.

At the moment I'm trying LordJashin's method, so I'm unsure at the moment which steps this could cover. I've never used DTM, so to prevent stupid mistakes I'm learning the basics of it before saying it will/won't work.

 

Last time I've played Runescape will be about 10 years ago, but I understand the map system hasn't changed much since. I suppose I could define something as textbox, and look for all places that look like a textbox. They are all the same size, it's only the text that differs. Is that what you're trying to say?

 

Not really what I was saying but I got you to say what you're trying to do and that was what I wanted I think...

 

If you're finding text all the time, and the text varies. I consider this very very hard. I think I remember asking you if the text overlaps, because if it does that can undermine the whole thing. I have 2 ideas then. You can either use FindBitmapMaskTolerance (haha its not just Tol yet), or the GetTextAt function. I think GetTextAt actually uses FindBitmapMaskTolerance.

 

[sCAR]

FindBitmapMaskTolerance(Maskbmp, var x, var y, x1, y1, x2, y2, Tol, ContourTol);

[/sCAR]

 

Returns the x, y of the found bitmap mask, and if it found it, it returns true ofc. The Tol, and ContourTol params are interesting. ContourTol I think is the Tolerance of the background color. Google says this:

An outline, esp. one representing or bounding the shape or form of something

 

A mask is the black and white version of something. Essentially if you make your own font set using the tutorial I posted before. In the tutorial you color the text all white, and black. White for the letters, and black for the background. That is so you can change they're color yourself in searching functions later. It comes in handy.

 

GetTextAt requires a bit longer example:

 

Here's a basic outline:

 

[sCAR]

program New;

 

var

Verdana: Integer;

 

begin

Verdana := LoadCharsFromFont2('Verdana', 12, false, false, false, false);

 

// or you could do

 

// LoadChars2('C:\YourDirectory\Thathas\YourPreMade\Font\UsingThatTutorial\OSIusesThisToo');

 

//GetTextAtEx(x, y, Tol, Chars, CheckShadow, CheckOutLine, MinSpac, MaxSpac, TextColor, TextLength, Strict, Range);

 

end.

[/sCAR]

 

Now, how to test all these functions out? Use Debug bitmap. Just as an example I will show you how to use both of these methods with that.

 

Alright I went pretty far. And just as an example of how hard this function is. I opened up note pad put in at Verdana at 12px:

 

Verdana := LoadCharsFromFont2('Verdana', 12, false, false, false, false); put that into notepad. GetTextAt only found:

 

Vedaa:=LadCasFomFon2

 

Lols, pretty good though to even find anything took me a couple tries. The coordinates have to be Exacto.

 

Open up notepad, and select it as client with the above and this script will show you what I mean. The 2nd Debug Bitmap shows u the SAME FONT and SIZE. OSI loads fonts from premade font sets using LoadChars2, but LoadChars2FromFont or w/e is what we're using atm.

 

[sCAR]

program New;

 

var

Verdana: Integer;

 

Testbmp: TSCARBitmap;

 

Client: TSCARWindowClient;

 

AnsiString1: AnsiString;

String1: string;

 

procedure LoadCrap;

begin

ClearDebug;

Verdana := LoadCharsFromFont2('Verdana', 12, false, false, false, false);

Testbmp := TSCARBitmap.Create('');

end;

 

procedure DebugIt(Bitmap: TSCARBitmap);

begin

DebugBitmap(Bitmap);

wait(3000);

HideDebugImgWindow;

end;

 

procedure SetupClient;

begin

Client := TSCARWindowClient(GetClient);

// for some reason I could not get the FindWindow API to work with the Debug Image Window

// So you need to target NotePad instead with crosshairs

end;

 

procedure FreeEverything;

begin

HideDebugImgWindow;

Testbmp.Free;

//Client.Free;

// Keep getting active client can't be freed errors

end;

 

begin

LoadCrap;

 

Testbmp := CreateBitmapFromText('TestingThisFunction', Verdana, clRed);

DebugIt(Testbmp);

 

Testbmp := CreateBitmapMaskFromText('THIS IS A MASK of Verdana at 12px', Verdana);

DebugIt(Testbmp);

 

// Those are good functions to create bitmaps from Text, for your own observation

// Since GetTextAt uses a position we need to setup a client too

 

wait(1000);

SetupClient;

Client.Activate;

wait(1000);

 

//MoveMouse(0, 0); for testing where its searching at, at 0, 0

 

wait(3000);

 

AnsiString1 := GetTextAtEx(0, 0, 0, Verdana, false, false, 0, 0, 0, 13, false, tr_AllChars);

 

String1 := GetTextAtEx(5, 9, 0, Verdana, false, false, 0, 0, 0, 13, false, tr_AllChars);

 

 

WriteLn(AnsiString1); // GetTextAt returns an AnsiString from the function

 

WriteLn(String1);

 

String1 := GetTextAtEx(0, 3, 5, Verdana, false, true, 0, 10, 0, 20, false, tr_AllChars);

 

WriteLn(String1);

 

FreeEverything;

end.

 

[/sCAR]

 

You could also test the integrity of the FindBitmapMaskTolerance function too. Same example, use CreateBitmapFromText with the font, then search for it with that function.

Edited July 29, 2012 by LordJashin

[MarkD]

I've tried to read the text with GetTextAtEx, so far it gives a lot of rubbish.

I'm aware of the amount of work that lies ahead of me, yesterday and today didn't give me a lot of time I'm afraid.

 

As soon as I have anything that works even a little, I'll just post it. In the meantime, there is a lot of study and a lot of practice required to get this going

[LordJashin]

I'm actually working on a bot for a game atm I started yesterday. So I will be messing with GetTextAt too.

 

Just remember that:

1. Spacing matters

2. Position you put in x,y matters

3. Font, and some of the other variables matter somewhat

 

I think that x, y matters a lot though because I kept moving it around till it came out with less rubbish.

[MarkD]

One of the things I wanted to try was Janilabo's suggestion.

However, it's not Scar 3.35 compatible. One of things missing: procedure GetBitmapSize(const Bmp: Integer; out Width, Height: Integer);

 

Is there any replacement for this?

[LordJashin]

One of the things I wanted to try was Janilabo's suggestion.

However, it's not Scar 3.35 compatible. One of things missing: procedure GetBitmapSize(const Bmp: Integer; out Width, Height: Integer);

 

Is there any replacement for this?

 

Yes there is, Freddy fixed it too in 3.35.01 bug fix or w/e. That is an interesting post, I'll check it out right now.

[FHannes]

Yes there is, Freddy fixed it too in 3.35.01 bug fix or w/e. That is an interesting post, I'll check it out right now.

 

No I ddin't... I fixed TSCARBitmap.Resize...

 

@MarkD: Bitmap.Width and Bitmap.Height represent the dimensions.

[LordJashin]

My bad lol.

 

I think Janilabo's post is good. His function I think uses GetTextAt, then moves over to 30 or something then does it again. Does something like that to avoid the length problem.

[MarkD]

It took some time, but I finally figured out how to do the find text part. I won't let it search for the text, but for the textbox. DTM does the rest.

 

program FindTextBox;
var
TxtBox,x,y: Integer;

Procedure InitDTM;
Begin
 TxtBox := DTMFromString('78DA6334666260D807C448606F3E2F986684F' + 
      '219CD81F21750D5945AB1A1AA0903CAEF4755F3EB58148A1A007F' + 
      '110728');
End;

begin
InitDTM;
 if(FindDTM(TxtBox,x,y,0,0,1680,1050))then
  begin
   Writeln('Succes!')
   MoveMouse(x,y)
  end;
end.

 

So, still no OCR (I've been fighting with that for a few days, it's hard as hell...), but at least the program is getting some body now.

 

Now, I need to exclude the coordinates that have already been found. I haven't found a way to make multidimensional array's, so I guess I'll have to use one for x and one for y.

I'll need to watch out for writing crappy code here, so I'm looking for a clean solution for the code below:

 

program FindTextBox;
var
 TxtBox,x,y,i,t: Integer;
 ExclListX,ExclListY: array of Integer;

Procedure InitDTM;
Begin
 TxtBox := DTMFromString('78DA6334666260D807C448606F3E2F986684F' + 
      '219CD81F21750D5945AB1A1AA0903CAEF4755F3EB58148A1A007F' + 
      '110728');
End;

begin
InitDTM;
i := 0;
 for t := 0 to 5 do
 begin 
   if(FindDTM(TxtBox,x,y,0,0,1680,1050) and (not ((x = ExclListX[0] and y = ExclListY[0]) or (x = ExclListX[1] and y = ExclListY[1]) or (x = ExclListX[2] and y = ExclListY[2]))then
    begin
     Writeln('Found one!')
     MoveMouse(x,y)
     ExclListX[i] := x;
     ExclListY[i] := y;
     i := i +1;
    end;
 end;
end;
end.

 

That's one hell of an ugly if-statement (besides that, it gives me a type mismatch error so I probably messed up), and only works for the first 3 segments of both array's (while I need to check the entire two array's). It is only written to show you what it should do, not how it's supposed to be done.

 

Suggestions?

 

EDIT: It just hit me why it could say mismatch. That's probably because it wants to check something with a segment that isn't initialized yet Had the same problem in the past with vectors (C++), i'll look how I fixed that one.

 

EDIT2: Found it, cheated my way out of it by filling the first 20 segments with zero during init. I could do the same here, but if anyone knows a 'clean' solution, I'd really like to hear it

 

EDIT3: I'm a little afraid this FindDTM will only give me the same coordinates over and over again. I suppose I'll have to edit the FindDTM variables to make sure this doesn't happen, I'm not sure how at the moment...

Edited August 4, 2012 by MarkD

[LordJashin]

Well at least you got something started. For the arrays, there is TIntArray, and TIntegerArray types. I forget what the difference is between them though.

 

for i := 0 to high(intArr) do

 

That's one way to do it. Now, I think what you are doing is relying on the DTM, and then trying to make it get new points, and exclude old ones that have already been found. You must use SetLength(Arr, Length(Arr) + 1) or something similar if you add a value to the array. You could do everything in TPoints.

 

You must have one value that is the Index of the Array, and then another value that loops through the array and checks against the values at the Index. Example:

 

[sCAR]if high(arr) = 0 then Exit;

 

for i := 0 to high(Arr) - 1 do

for j := i + 1 to high(Arr) do

if Arr = Arr[j] then

dowhatever;[/sCAR]

 

Also note there is some array deletion functions. For TPointArrays there is TPADelete, over at the wiki.

[MarkD]

It took me some time to find out, but I think a TPA is the tool for the job here. It's an array of 2 dimensional records, so I can just store my excluded X and Y in it.

Unfortunately, the amount of documentation about arrays is limited. To quote the old manual:

SCAR supports less types than Pascal and for those who are familiar with Pascal - it doesn't support arrays (ok it actually does, I just didn't want to explain it here ).

 

Is there a way to make the lenght of a TPA variable (like vectors)? If no, I'll just have to reserve about 20 points in it. In the meantime, I'm trying to get things working.

 

EDIT:

I'm afraid an exception list will not do the trick. The script will find a DTM, do something, add it to list. Then it searches again, finds the same DTM, does nothing. Searches again, finds same DTM and does nothing again. It will not find a new DTM, it'll just make sure the same DTM isn't handled twice. The real solution would be to find a DTM, do something and go search again from the last coordinate + 1.

 

Why didn't I think of that earlier...

Edited August 9, 2012 by MarkD

[MarkD]

So, lacking the source of the FindDTM function, I just hacked my way around it to test the concept.

As long as there is no FindDTMEx, it's kind of tricky to exclude coordinates.

 

[scar]program FindTextBox;

var

TxtBox,x,y,t,h_index: Integer;

ExclList: TPointArray;

 

Procedure InitDTM;

Begin

TxtBox := DTMFromString('78DA6334666260D807C448606F3E2F986684F' +

'219CD81F21750D5945AB1A1AA0903CAEF4755F3EB58148A1A007F' +

'110728');

End;

 

begin

ClearDebug;

InitDTM; //Load DTM

SetLength(ExclList, 80); //reserve some playground

ExclList[0] := Point(0,0); //make sure there is at least something

//to prevent issues when searching

h_index := 0;

for t := 0 to 4 do

begin

if(FindDTM(TxtBox,x,y,0,h_index,1680,1050)) //and not already done

then begin

WriteLn(x);

WriteLn(y);

MoveMouse(x,y); //point to match

Wait(300); //for debug

ExclList[t] := Point(x,y); //add to exclusion list

h_index := y+1; //hacky way of excluding coord

end;

end;

end.[/scar]

 

Anyone suggestions about the exclusion? I kind of feel guilty to hack my way around instead of writing clean code.

How does the FindDTM function does it's job?

[FHannes]

A new DTMs api is currently scheduled for SCAR Divi 3.37. It's possible it'll be in 3.36, but unlikely. I'm not sure there's a way for you to exclude points from the search with the current system.

[Bixby Sayz]

Grab the client screen to a bitmap, blank out the portions you don't want to search, set the client to the bitmap, do your search, set client back to original?

 

Would this work? Just thinking aloud. Didn't take the time to explore it.

[FHannes]

Grab the client screen to a bitmap, blank out the portions you don't want to search, set the client to the bitmap, do your search, set client back to original?

 

Would this work? Just thinking aloud. Didn't take the time to explore it.

 

It would actually, it's probably also the fastest way of doing it.

[MarkD]

That's a very good idea, changing the color on the coords of the found DTM would eliminate the possibility of finding it again (since it obviously no longer exists).

It would be wonderful if the DTM api would be more extensive, but I'll just try to make it work before that. We'll see who's quicker, but I guess Freddy wins

 

As usual, I'll keep you guys posted.

[MarkD]

I'm encountering two difficulties:

 

1. SetTargetBitmap() is removed (not surprising since SCAR is more and more working with objects) and I haven't found a substitute so far.

2. I'm getting an 'Argument out of bitmap bounds' runtime error on line 38 (Bmp.Pixels[x,y] := clRed), but perhaps this is caused by problem nr. 1.

 

I've seen Freddy's video about Bitmaps a few times, but I guess I either missed it or it's not in there.

 

[scar]program FindTextBox;

var

TxtBox,x,y,t: Integer;

List: TPointArray;

Bmp: TSCARBitmap;

Cl: TSCARClient;

 

Procedure InitDTM;

Begin

TxtBox := DTMFromString('78DA6334666260D807C448606F3E2F986684F' +

'219CD81F21750D5945AB1A1AA0903CAEF4755F3EB58148A1A007F' +

'110728');

End;

 

Procedure InitBMP;

Begin

Bmp := TSCARBitmap.Create('');

Cl := GetClient;

Bmp.Assign(Cl);

//SetTargetBitmap(Bmp);

end;

 

begin

ClearDebug;

InitDTM; //Load DTM

InitBMP; //Load BMP

SetLength(List, 40); //reserve some playground

List[0] := Point(0,0); //make sure there is at least something

//to prevent issues when searching

for t := 0 to 4 do

begin

if(FindDTM(TxtBox,x,y,0,0,1680,1050))

then begin

WriteLn(x);

WriteLn(y);

MoveMouse(x,y); //point to match

Wait(300); //for debug

Bmp.Pixels[x,y] := clRed; //paint it

List[t] := Point(x,y); //add to list

end;

end;

end.[/scar]

[LordJashin]

Sounds like some good ideas are going. For the change the pixel thingy. Just change the pixel of where the DTM is found, and that's it. You could do like...dtm find then change the search area to search from that DTM and forward. I like the 1st way better though, faster too I think.

 

You can do GetClient.Capture or Client.Capture to get a bitmap of the client...TSCARBitmapClient

[Janilabo]

Does this work(?):

 

[scar]program FindTextBox;

 

var

TxtBox, x, y, t, w, h: Integer;

List: TPointArray;

Bmp: TSCARBitmap;

Cl: TSCARClient;

 

procedure InitDTM;

begin

TxtBox := DTMFromString('78DA6334666260D807C448606F3E2F986684F' +

'219CD81F21750D5945AB1A1AA0903CAEF4755F3EB58148A1A007F' +

'110728');

end;

 

procedure InitBMP;

begin

Bmp := TSCARBitmap.Create('');

Bmp := GetClient.Capture;

Cl := SetClient(TSCARBitmapClient.Create(Bmp)); // New SetTargetBitmap(Bmp);

end;

 

begin

ClearDebug;

InitDTM; //Load DTM

InitBMP; //Load BMP

SetLength(List, 40); //reserve some playground

List[0] := Point(0,0); //make sure there is at least something

//to prevent issues when searching

for t := 0 to 4 do

begin

if FindDTM(TxtBox, x, y, 0, 0, (Bmp.Width - 1), (Bmp.Height - 1)) then

begin

WriteLn(x);

WriteLn(y);

MoveMouse(x, y); //point to match

Wait(300); //for debug

Bmp.Pixels[x, y] := clRed; //paint it

List[t] := Point(x, y); //add to list

end;

end;

SetClient(Cl).Free;

Bmp.Free;

FreeDTM(TxtBox);

end.[/scar]