1

I need to 1.) output and store a LIST of websites using BeautifulSoup the package. My results are too long: Official site: www.vigeland.museum.no/en/vigeland-park. 2.) How can I translate the type 'bs4.element.Tag' to list (basically)

At best, I only need 'www.vigeland.museum.no' etc

import requests # library to handle requests
from bs4 import BeautifulSoup as bs
#
r = requests.get('https://www.planetware.com/tourist-attractions-/oslo-n- 
osl-oslo.htm')
soup = bs(r.content, 'lxml')
print('request successful')
#
web_site=soup.find_all('div', class_="web")
for web in web_site:
    print(web.text)
    type(web)


### My RESULT ###
Official site: www.vigeland.museum.no/en/vigeland-park
Official site: www.khm.uio.no/english/visit-us/viking-ship-museum/
Official site: www.nasjonalmuseet.no/en/
Official site: http://munchmuseet.no/en
Official site: http://www.kongehuset.no/seksjon.html?tid=28697
Official site: www.khm.uio.no/english
Official site: http://frammuseum.no
Official site: www.skiforeningen.no/en/holmenkollen
Official site: https://www.oslo.kommune.no/politikk-og-a 
dministrasjon/radhuset/visit-the-oslo-city-hall/
Official site: www.akerbrygge.no/english
Official site: www.nhm.uio.no/english/
Official site: http://operaen.no/en/

bs4.element.Tag
0

A tidier and more efficient way would be to use class and child combinator to get the child a tag. You then get exactly the right number and no need for string tidying.

import requests 
from bs4 import BeautifulSoup as bs
r = requests.get('https://www.planetware.com/tourist-attractions-/oslo-n-osl-oslo.htm')
soup = bs(r.content, 'lxml')
links = [item['href'] for item in soup.select('.web > a')]
0

Split() the text value and then strip() white spaces and store only last value of string.

import requests 
from bs4 import BeautifulSoup as bs
r = requests.get('https://www.planetware.com/tourist-attractions-/oslo-n-osl-oslo.htm')
soup = bs(r.content, 'lxml')
print('request successful')
web_site=soup.find_all('div', class_="web")
websiteofficial=[web.text.split('Official site:')[1].strip() for web in web_site]
print(websiteofficial)

Output:

['www.vigeland.museum.no/en/vigeland-park', 'www.khm.uio.no/english/visit-us/viking-ship-museum/', 'www.nasjonalmuseet.no/en/', 'http://munchmuseet.no/en', 'http://www.kongehuset.no/seksjon.html?tid=28697', 'www.khm.uio.no/english', 'http://frammuseum.no', 'www.skiforeningen.no/en/holmenkollen', 'https://www.oslo.kommune.no/politikk-og-administrasjon/radhuset/visit-the-oslo-city-hall/', 'www.akerbrygge.no/english', 'www.nhm.uio.no/english/', 'http://operaen.no/en/']
  • Your answer was just perfect. I had 2 missing websites that should be at Number 2 and Number 9. I simply added them using: #websiteofficial.insert(1, 'wikipedia.org/wiki/Akershus_Fortress') #websiteofficial.insert(9, 'wikipedia.org/wiki/Oslo_Cathedral') – moski Jun 7 at 12:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.