PowerShell: Invoke-WebRequest and URL links

I was working on a project recently that needed to count the number of links on a given URL. With the code complete I wanted a simple way to validate my work to make sure my totals matched. So, I turned to Windows PowerShell for a quick test and to continue my learning of this powerful tool.

PowerShell 3.0 introduced the Invoke-WebRequest cmdlet that makes it easy to get content from a URI passed in as a parameter. The cmdlet goes out to the internet and returns collections of forms, links, images, and other significant HTML elements from a web page.

To run the command all you need is a webpage address as a parameter:

Invoke-WebRequest

From the image above you can see there is a lot of information returned. We can see the StatusCode, Content, Forms, etc. Each of these properties returned is also available for further inspection. To see a full list we can use the Get-Member cmdlet to retrieve them all. The cmdlet also allows for the “gm” alias to be used.

Invoke-WebRequest -Uri 'http://www.lego.com/en-us/' | gm -MemberType Property

We can see from the list below that all of the properties returned matches the content in the image above:

Invoke-WebRequest Properties

From the list I see the Links property is available which is exactly what I need for my code validation. However, just adding .”Links” to my Invoke-WebRequest command will only return the hyperlinks themselves. This might be all you need if you want to ensure a returned webpage contains a link you are looking for. However, since I need a sum of links I will append the Count method so I can see the final total.

Invoke-WebRequest URL Link Count

To see a parameterized version of this script visit my .

If you have solved any problems with a cool PowerShell hack please leave a comment below.

Related Articles:

Publishing scripts to TechNet

London Azure Camp slides and demos

London .NET Developers group logo

On Saturday November 15, 2014 I was invited to speak at the London Azure Camp hosted by the London .NET Developers group in London, Ontario, Canada.

The group is led by Tom Walker (@Tinytoot) and his team of volunteers as they try to spread the word about everything .NET in the City of London.

Alongside Tom and myself, there were presentations by a trio of Microsoft MVPs: Lori Lalonde (@loriblalonde), Andrei Marukovich (@amarukovich) and Julien Stroheker (@Ju_Stroh).

Tom invited me to speak on both Microsoft Azure Mobile Services and Microsoft Azure Media Services.

My slides are based on the Azure Readiness Content provided my Microsoft. However, these slides have been scoped down to my session and the notes field of each slide has been expanded with additional content.

I have published my PowerPoint slides as downloadable files through Dropbox: Cloud Powered Mobile Apps with Azure | Build end-to-end video experiences with Azure Media Services.

They are also available for viewing and download through SlideShare: Cloud Powered Mobile Apps with Azure | Build end-to-end video experiences with Azure Media Services.

As well, I have zipped up my demonstrations and added them to Dropbox. In this file you will find the step-by-step notes to re-create the demos plus all of the code snippets. The zip file also contains the PowerPoint files listed above.

You can also find these links as well as more information about my previous public speaking engagements under my Presentations page.

Please contact me if you have any follow up questions from the event. My contact information is also available within the slides.

Publishing scripts to TechNet

I have been experimenting with PowerShell for a while now and I have come to the conclusion that it is a very powerful tool. I like being able to manipulate various objects in the operating system so that I can automate everyday tasks.PowerShell Logo

One of the things I do a lot of is search my system for files. Normally I look by name but sometimes I need to search by file extension. So, I thought this would be the perfect candidate for a script.

Now, this blog post will not go into great detail on what this script does. If you want to view it or download it you can find it in my .

Similarly, I will assume you know how to run scripts in PowerShell. If you have never used PowerShell then now is the time to learn! There are lots of tutorials on the web. You can start with the Windows PowerShell User’s Guide.

Instead, this post is about how you can contribute scripts to TechNet on the Microsoft site. If you are unfamiliar with TechNet consider it a portal containing information and technical resources for Microsoft products.

 

TechNet Galleries

There are two areas on TechNet that you can post to. One is the TechNet Gallery. This space includes scripts as well as resources for all of the current Microsoft operating systems.

The other is the TechNet Script Center Repository. This is a section of TechNet dedicated solely to scripts written in VB Script, PowerShell, SQL, JavaScript, or other scripting languages.

Upon reading up on the differences between the two it seemed that in previous years you needed to post a script to both sites individually. However, my experience was that posting to the Script Center Repository replicated it to the TechNet Gallery.

If you want some background on why it is prudent to have your script in both galleries you can check out this article entitled How to Publish your Script to Both the TechNet Gallery and the TechNet Script Center Repository. Author Craig Lussier says that by “having your script listed in both the Gallery and Script Center Repository you can ensure that there are more access points for your contribution for the community at large.

Before you can begin inserting your script you must have a Microsoft account. You can sign up for one on the TechNet site. This account can be used here and is also the same account you would use on the MSDN forums if you have questions. For those of you who have not been using TechNet or MSDN lately you will also see that Microsoft has added a badging system like StackOverflow’s to your profile. You can see an example on my profile below:

 

How To Upload

Once you are logged into TechNet you will see an area on the left-hand side entitled “Quick Access.” Under this title there will be an “Upload a contribution” link. Click it to begin the process.

Use the “File upload” section to browse to your PowerShell (*.ps1) or other script file. Then give your contribution a title that describes your script. For mine I am calling it “Get files by extension within a given path (PowerShell).” I added the script type to the header so that it is clear within the search results what type of script it is.

Next, use the “Description” area to provide more information about your contribution. You can add code snippets, screenshots, additional files and even HTML to illustrate your script. To see an excellent example of a description with HTML, images and code snippets check out this highly rated contribution.

TechNet Upload area

For my description I am adding a summary of what the user should expect when they run the script. I am also adding my full PowerShell file using the “Insert Code Section” button on the description toolbar. To do this I just needed to select my script language and then paste my code into the left-hand side. A preview is automatically generated on the right-hand side for me. If you are happy with the preview click the “Insert” button to add paste it in.

TechNet Insert Code Section

The “Summary” is what will be displayed in the search results. You can default to using the initial 280 characters of your description or you can write a new one. The site will raise an error if your description has less than 280 characters though. You will also need to set the language of your description.

TechNet Upload area

Choosing a Category was tricky for me as the script is more of a productivity item. I searched the Script Center Repository for similar scripts to see which category others chose. I eventually went for the Storage category. I then chose the Files sub category.

Tip: To find a category you could always browse through the sub categories first to see if a narrower topic can help define your top-level category.

TechNet Upload area

Now set the “Operating systems” that you have tested the script on. You can then assign tags to the submission so that it is easier to find and categorize. The list is auto-populated so you can type in as many as you need. Again, looking at similar scripts in the repository can help you choose tags that are appropriate.

Under “Options” you can then elect to have the Questions and Answers tab on your contribution. One of the benefits of adding your script to TechNet is that people can comment on it and ask questions. I have also seen commenters provide alternate ways of accomplishing the same tasks which allows you to learn even more.

Next, choose the “License” that appeals to you and then accept the Terms of use. Take some time to read the terms. If this is proprietary code then this section might be of particular interest:

 

We do not claim ownership of any Submission unless otherwise agreed to by the parties. However, by providing a Submission, you are irrevocably granting Microsoft and its Affiliates the right to make, use, modify, distribute and otherwise commercialize the Submission in any way and for any purpose (including by granting the general public the right to use your Submissions in accordance with this Agreement, which may change over time).

Finally, if you are happy with the results click “Save and publish.”

TechNet Upload area

 

How The Script Appears

Clicking “Save and publish” makes the script appear instantly in the TechNet Script Center Repository. Once published the contribution URL can be found under “My contributions” in the “Quick Access” area.

A quick search of both the script name and of my last name in the TechNet Script Center Repository confirms that the script is now live. A similar search in the TechNet Gallery also displays my script.

So, officially my Get files by extension within a given path (PowerShell) script can now be found at .

One interesting thing to note is that I now have an additional link under the “Quick Access” area called “Browse script requests.” This link lists scripts that were requested by other community members. This will definitely be a great starting point for future PowerShell project ideas.

 

Conclusion

As you can now see I have a PowerShell script in both GitHub and the TechNet galleries. The whole process was very intuitive and quick. I encourage you to think about contributing your scripts to TechNet.

Please download my script and try to run it on your system. As well, please leave a question or a rating on a TechNet gallery page.

Hopefully you have found this blog post useful. If you have any questions or comments please leave me a message below.

Speaking at ConFoo 2015

I am super excited to announce that I have been accepted as a speaker for ConFoo 2015 in Montreal this February. This is a major conference that attracts presenters and attendees from across North America and Europe.

ConFoo bills itself as a “multi-technology conference for web developers.” This is a motto they live up to as they will be covering 150 diverse topics in just three days.

This is the first time that I applied to speak and fortunately two of my submissions were accepted. My first session is entitled An introduction to Universal Windows Apps. The second is on Building high-performance software with Application Insights. Both are relatively new technologies so I’m hoping that there will be high interest among attendees.

And, since I am a speaker, I can now flash this badge on my website:

ConFoo. February 18 - February 20, 2015 | Montreal, Canada

If you are planning to attend what should be an amazing ConFoo, then please feel free to introduce yourself in the comments section below or at the conference. I look forward to speaking with you.

Related Articles:

10 tips on submitting a conference session proposal

My Pluralsight video is now live

For those of you who have been following my most recent posts (My adventures with Pluralsight, My adventures with Pluralsight – UPDATE) related to my Pluralsight experiences, I am happy to say that my AngularJS video with John Papa has now been published!

The video is entitled Play by Play: Learning AngularJS With Ken Cenerelli and John Papa.

The description reads: “2014 brought Pluralsight’s 10th Anniversary, and to celebrate, we gave one of our fans a trip to film a Play by Play with one of our top authors for a pair programming session. This session is an excellent beginner-level introduction to AngularJS hosted by John Papa.

The video was announced with this tweet on October 8, 2014:

Another tweet on October 17, 2014 resulted in a plethora of favorites and re-tweets:

The 90-minute tutorial currently sits at number 12 in the top 100 most popular courses over the last 10 days. It has a rating of 4.5/5 which translates to a “Very Good.”

The video seems to be helping people as this tweet attests:

You will need a Pluralsight subscription to view the entire video. However, Pluralsight does offer a free 10-day trial.

If you wish to view the introduction you can see it here on YouTube: Play by Play: AngularJS tutorial for beginners.

I encourage everyone to check it out. Hopefully you will see the fun I had in making it and learn something about AngularJS along the way!

Follow

Get every new post delivered to your Inbox.

Join 84 other followers