Jaló Chris! This is an interesting task that you are describing. I have checked the first part of the whole process and it seems pretty impossible to automate it unless you do a small trick. Let me explain the problem first and then how you can work around it:

The obvious problem is the captcha image. However, we can probably make our job wait till you manually enter the captcha if there is one (or prompt you to enter one). The real problem here though is the following: WinAutomation has no way of knowing how many items will appear in the screen and if the number that appears in the "global monthly ... blah blah..." is higher than 1000. And even if it did know, there is no clear way on how to press the appropriate "Add" button.

The work around that I thought about includes the following steps:

1) WinAutomation reads the list of keywords from a text file
2) Loops through each keyword doing the following:
a) navigates to the keyword search tool page
b) sends the current keyword to the appropriate field
c) if there is a captcha, save it locally and launch it with Windows picture and fax viewer prompting the user to enter its value
d) submit the keyword and wait for the results (a specific image needs to be picked that will verify the end of the search)
e) Send a Ctrl+F keystroke combination
f) Send the exact keystrokes: Download all keywords: text, .csv (for excel)
g) This will select the specific part of the screen so "Move Mouse to Image" to move the mouse at the (for excel) part of the screen which is highlighted
i) Send a mouse click, wait for window to open and save file locally

j ) THEN, and ONLY THEN, we will open the saved csv file with Excel (can be done with "Read Text from File" too but I prefer Excel since it is easier to modify), and remove all the lines that have a number in their 4th column that is smaller than 1000.
h ) Save the modified file (that now only contains the keywords that we want) and move on to the next keyword (step <a> ).

What do you think of this solution for the first part of your task?
 

Theres an easier way...

You can get around teh captcha image in the google keyword tool by using SEObooks KW tool. It still queries the google DB, but they dont require you to enter a captcha. :)

Im very interested in the second part of this...find competition numbers without a captcha.

All the best

Barry

Well, I can't throw any ideas unless I have a concrete URL or screenshot to understand what buttons need to be pressed and what is the interface like.

Depending on the way that the website has been built, results that may seem "not saveable", can be easily extractable through the page's source code. I will still need a URL or the page saved as an HTML file to determine how easily can the data be extracted. This second part could be automated (I will be sure if I have some sample URLs to examine) through "Download from Web" action (which means that it will run in the background).

Here's a link: https://adwords.google.com/select/KeywordToolExternal

This job would be VERY POPULAR for anyone who does any kind of online research requiring keyword research (as google is the keyword research tool of choice for most people).

Also it's important to note that if a user is logged in, you won't have the captcha issue (you can sign up quite easily and free to get an account, perhaps it may be helpful to build the job with and/or without the captcha). 

In the lower right you see the option to save the text as csv or as txt.  Looking forward, if you save as csv and wanted another job that could say sort the text based on highest to lowest word count, cpc, etc that would also be possible. I suppose txt format is good for reading line by line, etc. 

This job would work really well to run together with Jag's recently saved jobs as well!

This would be a great job to have and I bet alot of people would be attracted to winAutomation if we could get this out. I bet I could get a video up on how to properly use the job if we could get this done this week.

 popular tools used:

https://adwords.google.com/select/KeywordToolExternal

http://www.google.com/sktool/#

http://freekeywords.wordtracker.com/

Is there a way to scrape google kw external using Download From Web action?

The "Download from Web" action pretty much acts like a browser with Javascript disabled. If you try to get results from the Google kw external page with JavaScript disabled in your browser, you will see that you can't. This is a very fast way to determine which tasks can and which tasks cannot be automated with the download from web action.

P.S: In order to disable JavaScript in Mozilla Firefox go to: "Tools - Options" and under the "Content" tab deselect the option "Enable JavaScript"

I wonder how do other people scrape data from Google Keyword Tool?

THIS INFO WOULD BE SO HELPFUL - ANYONE MAKE ANY PROGRESS WITH SCRAPING THE ADWORDS TOOL?

NO IDEAS ANYONE?? THIS JOB HAS GONE UNANSWERED FOR LIKE 3 MONTHS!

Almost have it and I think it will be possible with winautomation
A lot of problems to over come with this, I am at the point of finding the (Inz Tuel) data, and google bans my ass each time, (And if you look at these results google is Geo targeting here so your location will affect the results this I never seen before) I now have a list of about 600 data centers which I thought would allow me to pull this data fast without getting my IP banned.  But no even pulling each result from a different data center I still get banned. When I have time I will work on it some more

 As I 'm kind of new to this forum, I came across this thread today and it does not seem like a very challenging task to me... I have attached a sample job that will do the following:

1) Check whether there is a captcha image included:
a) If there is captcha, prompt the user to enter the captcha content
b) If there is no captcha, it will directly input the keyword

2) Enter the keyword

3) Press the "Get Keyword Ideas" button

4) Wait till the ideas are fully loaded

5) Save the produced csv file

There are of course some things that need to be altered:
(I) The images that I captured may not work for your machine, so you may need to recapture them
(II) You will probably want to read the keywords from a text file, so a for each loop needs to be added (it should include actions no.3 to the end of the job).
(III) You should add a variable or the keyword name in the csv's file name

Other than that, I tested the job and it works absolutely fine.

Hi There. First, you make some great forum contributions - thanks! Wish more people could be as active as you :S

To speed up the job, I usually log in - a) it makes jobs easier because there's no captcha b) you get access to other data c) it's free

What I try to do in most of my posts are ask about jobs I know people are doing by the thousands each day and would love to automate. NO ONE DOES ONLINE KEYWORD RESEARCH WHO DOESN'T USE THIS TOOL...

That said, I would prefer to remove the human element and the pressing of the key means one can't go and come back - or can you?
 

For another job I have, I get data similar data using download from web and schedulers. It is absolutely fantastic because several times a day, data is downloaded, parsed, analyzed, then sent to a few people - ALL IN THE BACKGROUND. The only thing that happens is that at the end of the day I see the emails notifying me that the jobs were done and were sent to the right people. Keep in mind ALL THIS WAS DONE BY HAND BEFORE.

Unfortunately, until we can remove the human element, it won't be able to work like this I feel.

THANKS AGAIN FOR YOUR CREATIVE SOLUTIONS!

That's strange - are you logged in?! IF you are logged into adwords then you don't get the captcha. Also it'sfree too - only takes a second to sign up. BE AWARE THAT THERE'S A WHOLE HOST OF MARKET INFO THAT YOUCAN'T SEE IF YOU ARE NOT LOGGED IN - The "G" has actually been abit quiet about it, but it's great info...

I have been in the automation area for more than 2 years now, but I still remember my first macros and scripts and the difficulties that I faced. Thankfully, there were some very good people in WinAutomation's support department that helped me in my first steps, and I saw this forum as an opportunity to give something back. 

There is no human element involved here if there is no captcha image in the page. I have added the condition for the captcha image because I did not get consistent results about when the captcha appeared and when it did not appear. That said, the job will automatically identify whether there is a captcha image, and if there is one, it will prompt the user to enter its content (you can of course modify it to automatically submit the captcha to a decaptching service).

Also, the last action in the job ("Send Keys") sends the following keystrokes:
Alt+D: This is a hotkey that works in all browsers and selects the address bar
Enter: Goes to the selected URL (which is the initial page of the keyword tool)

This combination will take you to step no.1 of the job, and this is why you can easily include in a loop actions no.3 to no.31

You bet! I'm going to take a look at this - thanks. BTW, I've figured out a pretty entertaining way to enable REMOTE CONTROL AUTOMATION with Winautomation from a mobile phone - pretty cool, huh? Shoot me a PM maybe we can connect over IM. It's actually quite fun to creatively come up with ideas that save so much time.

Take care....

Hey everyone

I have a solution to the problem of getting pass the captcha and straight to the keyword tool.

Im not really new to this software, i got to try it a while back but never got around to buying it until last week. The attachment is below and there are a few things i must tell you in order for this to work for you:

1) i used firefox...sorry its just better and faster than i.e to me

2) I have a google adwords account. Its free to sign up and is what made this process super easy to set up

3) An edit you need to make to the job: in step 4 you need to replace the text that says "your email" and "your password" to your email and password. This logs you in to google adwords.

4) I made the last step to switch it to the old interface. Its better than the new one to me.

Now that all of this is out of the way, we just need a way to tell the program what words to search for via excel. Im hoping for my work, someone posts that, so we can put it all together and have a great automated tool.

Ill be watching this thread, and thanks for the encouragement

Sampson

**An Update**

Hi everyone

I have updated my tool, to where it asks me for the root keyword i want to search on, then it goes to google, grabs the keywords and opens the file in .csv format.

I was wondering if there was a way to grab all of the keywords from colum a, and save them to a list to go back and grab more keywords for.

My next idea(and what i normally do in keyword research) Isplace all of the keywords i got from my root keyword into the goolge keyword tool to get the max possible keywords for my specific subject. Then i will save them all, remove duplicate and bad words, to complete my list.

If anyone runs across ideas to do this, it would be truly appreciated.

Thanks

LaPrentiss

 I think there is a dedicated action to do that. It allows you to convert a data table's column into a list. Then you can just loop through each item in the specific list.

guys, I'm also looking for such a tool but want that it also can input urls and search for which keywords google find with their searches instead input words. I have lists with more as 10k urls to check and so that is save all founded keywords and their search counts in a file. Is that possible and can some create that or tell me how to do? I need to start learning this tool and then it is maybe to complicated or not?

is this thread dead or are here still people that want to help me with above questions?

Why does nobody help?

Hello Buddym,

I think you could do some searching on http://www.google.com/sktool/#
where you enter the website you are interested in promoting and type in a keyword or more..

Not quite sure whether this is what you are after..

Samantha

 Any updates on this thread?  The jobs which are most interesting are:

- Enter a seed keyword into GKT, run (possibly with filters), export and download the .csv

- loop through the keyword phrases in the .csv, enter this as a seed keyword to the GKT, download the .csv, repeat.  

- After an arbitrary amount of cycles, combine all into one spreadsheet, remove duplicates

This thread is why I joined the forum and also why I will be downloading the trial.  If I can get the software to perform this task, my only other purchase desicion would be based on support levels.