Automated use of SBuilderX via external scripting

This forum is dedicated to discussions about the SBuilderX update for 64-bit Windows platforms.
Post Reply
pcpilot2010
Posts: 5
Joined: Tue Feb 17, 2015 5:47 pm

Automated use of SBuilderX via external scripting

Post by pcpilot2010 » Thu Feb 19, 2015 2:00 am

Hello Luis and all the regular members of this forum, Greetings !

I just became a member here and I am relatively new to SBuilderX and FSX SDK in general; yet my ideas are a bit ambitious, given my ignorance :-) I have a question, and hope you will be able to advise (please redirect me to the proper post if this has already been addressed, I couldn't find anything based on my search).

I am interested in making photo scenery on a somewhat larger scale (e.g., the north-eastern region of the US). Doing this manually would be tedious. So, I am trying to automate the communication with SBuilderX using the AutoIt scripting tool (with internal logic for selecting the blocks). Seems to be quite feasible in my early versions (though of course manual intervention cannot be avoided along the coast line, for water masking and blend masking; but even these can be partially automated - other than the actual construction of the coastline). A basic requirement here is that I should download image blocks according to a well defined grid pattern, with proper file naming convention indicating the block numbers (with a resolution that one may choose). I am shooting for an LOD of 15. So I select (rather the script selects, based on its internal calculations) a center point and a magnification of 13 on the main screen of SBuilderX. Then the "add map from background" screen is activated and the resolution is moved up three notches to magnification 16 (88x56 tiles in total, generating a bmp block of a little over 1GB). This works perfectly well and I have implemented stuff to take care of things even when SBuilderX does not behave as expected (involves conservative timers, so more time would be wasted than doing it manually). What I get each time is a block with aspect ratio of 11x7 (from within the script I do not have the ability to move the mouse precisely and select other aspect ratios that I may want - e.g., a symmetric 8x8 which would make file naming easier). So, my question is, does SBuilderX offer any program line options or API to directly specify a block in terms of a NW and SE corner (rather than going through the GUI). In general does SBX expose any of its other internal states, in which case I can do things in a less clumsy and quicker way than the way I am doing it now (with long timers).

By the way, if this capability is not old hat, and is of interest to others I will be most happy to share it here whenever I am able to complete...

Thank you, Luis and everybody.

Gopal

User avatar
luisfeliztirado
Posts: 436
Joined: Sun May 15, 2005 9:15 am
Location: Santo Domingo

Re: Automated use of SBuilderX via external scripting

Post by luisfeliztirado » Fri Feb 20, 2015 12:10 am

Hello,

This is a fascinating and very clever improvement on the process for creating photo-real scenery. Congratulations.

I don't remember ever having seen anybody else try something of the sort - you seem to be the first.

To answer your question, I don't think there are any hooks or other means available to you for accessing SBuilderX routines. Only Luis Sá, the creator of the program, could give you a definitive answer, but in all the years I have been working with him and his program, I have never come across anything of the sort.

Best regards.
Luis

pcpilot2010
Posts: 5
Joined: Tue Feb 17, 2015 5:47 pm

Re: Automated use of SBuilderX via external scripting

Post by pcpilot2010 » Fri Feb 20, 2015 10:11 pm

Hi !

Thank you very much for you reply and encouraging comments. It is ok, I can make this work even by interfacing directly with the GUI, though this may be a bit slower. I already have the basic structure in place, to simply download seamless scenery blocks in batches and create bgl - without involving any water-masking that is). I am currently working on fine-tuning the ability to handle all situations involving SBuilderX (e.g., if it gets frozen up then the script should detect this, terminate the process and reactivate from the last successful block, etc.) . I will share an early version hopefully soon.

I intend to use a bgl file naming convention of the form X.Y.LOD where X identifies the latitude block and Y identifies the longitude block. For example, if I shoot for an equatorial LOD of 15, it translates to a magnification of 16 on SBX, and each of my basic blocks will be comprised of an 88x56 tile matrix; X will range from 0 to 1490 and Y from 0 to 2341. For other LOD targets the file indices will scale accordingly ( mix LOD's in a contiguous region may get to be tricky though). This way the user does not have to be very precise in specifying a target rectangle; the program can check if a block within exists on the disk already, in which case it will be skipped. On the other hand when water-masking needs to be done along the coast the user can force overwriting with a switch.

I also have some thoughts on enabling water-masking via GIMP 2 with partial assistance, where the user only needs to handle the channel or magic eraser tool and then click a button on my Msgbox, when the masks will be saved (perhaps the blend mask can be auto generated by an internal program, based on a specified distance from the boundary of the water mask). I need to find an easy way for the user to generate a poly-line along the coasts (by simple mouse clicking) which can be read by the tool. The sequence of downloaded bmp blocks will be successively presented (prior to bgl) for user intervention on GIMP 2 (with the option to take breaks and suspend action mid-way at will). I am also trying to see if it is possible to implement an "ocean detection capability" (possibly based on comparing the RGB intensities and uniformity) - with this, if a user spills over to the ocean while specifying a rectangle, then only the relevant blocks over land will be downloaded.

Any way, lots of crazy ideas, not sure to what extent any of them is feasible :) Will keep everybody posted.

Regards.

Gopal

User avatar
luisfeliztirado
Posts: 436
Joined: Sun May 15, 2005 9:15 am
Location: Santo Domingo

Re: Automated use of SBuilderX via external scripting

Post by luisfeliztirado » Sat Feb 21, 2015 12:21 am

It all sounds very good and should make the entire process even easier.

The naming convention seem very appropriate. Please don't forget to consider the scenario where no tiles at all are available for download. That could easily lead to freezing..

I think you should be able to find water polygon shapefiles for most of the world to use as Water Masks. These shapefiles are often very accurate for the more advanced countries anyway, much less so for the rest of the world.

Please keep us posted on your developments and thank you.

Best regards.
Luis

pcpilot2010
Posts: 5
Joined: Tue Feb 17, 2015 5:47 pm

Re: Automated use of SBuilderX via external scripting

Post by pcpilot2010 » Sun Feb 22, 2015 12:11 am

Sure, I will update when I have version 0 ready, hopefully very soon; please stay tuned. I am yet to encounter an example with missing scenery. I will look for it to determine if additional procedures are needed to handle such cases (if you know of any lat/long combination, please advise). I didn't know that water polygon shape files were available. I will look for them; this opens up a whole new set of possibilities (theoretically, zero manual intervention !!!). Need to education myself on polygon intersection algorithms, etc.

Thanks and regards.

Gopal

pcpilot2010
Posts: 5
Joined: Tue Feb 17, 2015 5:47 pm

Re: Automated use of SBuilderX via external scripting

Post by pcpilot2010 » Wed Jul 08, 2015 6:54 am

Hello Luis, please check the message I have sent you. Thanks. Gopal

User avatar
roger-wilco-66
Posts: 92
Joined: Wed Apr 14, 2010 1:56 pm

Re: Automated use of SBuilderX via external scripting

Post by roger-wilco-66 » Tue Sep 01, 2015 6:23 am

Hi Gopal,

did you manage to continue with your brilliant (if I may add) idea? Sounds really good, a very interesting idea.


Cheers,
Mark
Development of historical sceneries for FSX and Lockheed Martin Prepar3D Simulators

My scenery development galleries:
https://www.dropbox.com/sh/x0skkam7xu8zz8r/DFwnonB1nH

pcpilot2010
Posts: 5
Joined: Tue Feb 17, 2015 5:47 pm

Re: Automated use of SBuilderX via external scripting

Post by pcpilot2010 » Tue Sep 01, 2015 12:51 pm

Hi Mark:

Thank you for your interest and kind words. Yes it works perfectly fine. But I have been reluctant to use it,let alone share it, because I am not sure of Google's policies in reference to bulk download. See, with this tool one can download significant amount of imagery without manual intervention in a matter of hours. I did come across one of their web pages stating "robotic download of imagery" being against their Terms of Use, or some such. I consulted with a few folks on FSDeveloper forums and at least one experienced member expressed concern about the potential legal tangle. So I have been wanting to try and re-deploy the tool based on some Governmental image sources, e.g., USGS for US imagery and their possible counterparts in other countries (instead of Google or Microsoft VE). But I got busy lately and that project extension is still pending :-)

Please share with me if you have more insights on a way of using Google or Virtual Earth on bulk basis, for personal enjoyment of FSX, without risking legal trouble. Is there perhaps a paid option (which still fits within our meager hobby budget) ?

Thanks.

Gopal
roger-wilco-66 wrote:Hi Gopal,

did you manage to continue with your brilliant (if I may add) idea? Sounds really good, a very interesting idea.


Cheers,
Mark

User avatar
roger-wilco-66
Posts: 92
Joined: Wed Apr 14, 2010 1:56 pm

Re: Automated use of SBuilderX via external scripting

Post by roger-wilco-66 » Wed Sep 02, 2015 7:20 am

Hi,

thanks for your notice.

Sorry to hear about the legal quirks.
Not sure about the google policies, I don't understand that legal problem entirely, I must confess. There are other applications that also work in a similar way or can read or capture the GE tiles. The other issue is that you can do the same step by step anyhow by hand. It is just more tedious to do all that manually. The result is the same, just the "bulk" work takes longer. I understand the problem of passing along of GE material to others, but in our case it is non commercial and can't be used in any other context than the sim due to the resampling. Not sure if this has any impact on the legal side, I'm not a lawyer.
Have you also looked into bing maps or nokia maps regarding license formalities ?

I'm sure if you pay for the service you can do what you please with the material, however, this is intended for commercial usage and therefore probably quite expensive. Our Freeware niche just doesn't seem to fit into these concepts.


Best,
Mark
Development of historical sceneries for FSX and Lockheed Martin Prepar3D Simulators

My scenery development galleries:
https://www.dropbox.com/sh/x0skkam7xu8zz8r/DFwnonB1nH

Post Reply